Determining control actions of decision modules

ABSTRACT

Techniques are described for implementing automated control systems that manipulate operations of specified target systems, such as by modifying or otherwise manipulating inputs or other control elements of the target system that affect its operation (e.g., affect output of the target system). An automated control system may have one or more decision modules that each controls at least some of a target system, with each decision module&#39;s control actions being automatically determined to reflect near-optimal solutions with respect to or one more goals and in light of a target system model having multiple inter-related constraints, such as based on a partially optimized solution that is within a threshold amount of a fully optimized solution. Such determination of one or more control actions to perform may occur for a particular time and particular decision module, as well as be repeated over multiple times for ongoing control.

BACKGROUND

Various attempts have been made to implement automated control systemsfor various types of physical systems that have inputs or other controlelements that the control system can manipulate to attempt to providedesired output or other behavior of the physical systems beingcontrolled. Such automated control systems have used various types ofarchitectures and underlying computing technologies to attempt toimplement such functionality, including to attempt to deal with issuesrelated to uncertainty in the state of the physical system beingcontrolled, the need to make control decisions in very short amounts oftime to provide real-time or near-real-time control and with onlypartial information, etc.

However, various difficulties exist with existing automated controlsystems and their underlying architectures and computing technologies,including with respect to managing large numbers of constraints(sometimes conflicting), operating in a coordinated manner with othersystems, etc.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a network diagram illustrating an example environment in whicha system for performing cooperative distributed control of targetsystems may be configured and initiated.

FIG. 2 is a network diagram illustrating an example environment in whicha system for performing cooperative distributed control of targetsystems may be implemented.

FIG. 3 is a block diagram illustrating example computing systemssuitable for executing an embodiment of a system for performingcooperative distributed control of target systems in configured manners.

FIG. 4 illustrates a flow diagram of an example embodiment of aCollaborative Distributed Decision (CDD) System routine.

FIGS. 5A-5B illustrate a flow diagram of an example embodiment of a CDDDecision Module Construction routine.

FIGS. 6A-6B illustrate a flow diagram of an example embodiment of adecision module routine.

FIGS. 7A-7B illustrate a flow diagram of an example embodiment of a CDDControl Action Determination routine.

FIGS. 8A-8B illustrate a flow diagram of an example embodiment of a CDDCoordinated Control Management routine.

FIG. 9 illustrates a flow diagram of an example embodiment of a routinefor a target system being controlled.

FIG. 10 illustrates a network diagram of a portion of a distributedarchitecture of an example CDI system.

FIG. 11 illustrates a network diagram of a portion of an example RulesBuilder component.

FIG. 12 illustrates a network diagram of example sub-components of anexample Rules Builder component.

FIG. 13 illustrates a network diagram of interactions of portions of aRules Builder component with an executing agent.

FIG. 14 illustrates a network diagram of interactions of portions of aRules Builder component with chattering components.

FIG. 15 illustrates a diagram of a sliding window for use withchattering.

FIG. 16 illustrates a diagram of using Lebesgue integration toapproximate a control trajectory for chattering.

FIG. 17 illustrates a diagram of various interactions of differentportions of a CDI system.

FIG. 18 illustrates a diagram of example different types of rules.

FIG. 19 illustrates an example user interface related tomedical/clinical auto-coding.

FIG. 20 illustrates a diagram of some components of a CU system.

FIG. 21 illustrates a diagram of performing Pareto processing for usewith mean field techniques.

FIG. 22 illustrates a network diagram of an example decision moduleagent.

FIG. 23 illustrates a network diagram of an example of offline workflowsfor knowledge capture.

FIG. 24 illustrates a network diagram of an example of workflows formean field computation and Pareto Optimal.

FIG. 25 illustrates a network diagram of an example of an automatedcontrol system for a home solar micro-grid electrical generating system.

FIG. 26 illustrates a diagram of workflow and components of a portion ofa CDI system.

FIG. 27 illustrates a diagram of a workflow for an inference processportion of a CDI system.

FIG. 28 illustrates a diagram of an overview workflow for a portion of aCDI system.

FIGS. 29A-29K illustrate examples of using a CDI system to iterativelydetermine near-optimal solutions over time for controlling a targetsystem.

DETAILED DESCRIPTION

Techniques are described for implementing automated control systems tocontrol or otherwise manipulate at least some operations of specifiedphysical systems or other target systems. A target system to becontrolled or otherwise manipulated may have numerous elements that areinter-connected in various manners, with a subset of those elementsbeing inputs or other control elements that a corresponding automatedcontrol system may modify or otherwise manipulate in order to affect theoperation of the target system. In at least some embodiments andsituations, a target system may further have one or more outputs thatthe manipulations of the control elements affect, such as if the targetsystem is producing or modifying physical goods or otherwise producingphysical effects.

As part of implementing such an automated control system for aparticular target system, an embodiment of a Collaborative DistributedDecision (CDD) system may use the described techniques to performvarious automated activities involved in constructing and implementingthe automated control system—a brief introduction to some aspects of theactivities of the CDD system is provided here, with additional detailsincluded below. In particular, the CDD system may in some embodimentsimplement a Decision Module Construction component that interacts withone or more users to obtain a description of a target system, includingrestrictions related to the various elements of the target system, andone or more goals to be achieved during control of the target system—theDecision Module Construction component then performs various automatedactions to generate, test and deploy one or more executable decisionmodules (also referred to at times as “decision elements” and/or“agents”) to use in performing the control of the target system. Whenthe one or more executable decision modules are deployed and executed,the CDD system may further provide various components within or externalto the decision modules being executed to manage their control of thetarget system, such as a Control Action Determination component of eachdecision module to optimize or otherwise enhance the control actionsthat the decision module generates, and/or one or more CoordinatedControl Management components to coordinate the control actions ofmultiple decision modules that are collectively performing the controlof the target system. Additional details related to such components ofthe CDD system and their automated operations are included below.

As noted above, the described techniques may be used to provideautomated control systems for various types of physical systems or othertarget systems. In one or more embodiments, an automated control systemis generated and provided and used to control a micro-grid electricityfacility, such as at a residential location that includes one or moreelectricity sources (e.g., one or more solar panel grids, one or morewind turbines, etc.) and one or more electricity storage and sourcemechanisms (e.g., one or more batteries). The automated control systemmay, for example, operate at the micro-grid electricity facility (e.g.,as part of a home automation system), such as to receive requests fromthe operator of a local electrical grid to provide particular amounts ofelectricity at particular times, and to control operation of themicro-grid electricity facility by determining whether to accept eachsuch request. If a request is accepted, the control actions may furtherinclude selecting which electricity source (e.g., solar panel, battery,etc.) to use to provide the requested electricity, and otherwise thecontrol actions may further include determine to provide electricitybeing generated to at least one energy storage mechanism (e.g., tocharge a battery). Outputs of such a physical system include theelectricity being provided to the local electrical grid, and a goal thatthe automated control system implements may be, for example, is tomaximize profits for the micro-grid electricity facility from providingof the electricity. It will be appreciated that such a physical systembeing controlled and a corresponding automated control system mayinclude a variety of elements and use various types of information andperform various types of activities, with additional details regardingsuch an automated control system being included below.

In one or more embodiments, an automated control system is generated andprovided and used to control a vehicle with a motor and in some cases anengine, such as an electrical bicycle in which power may come from auser who is pedaling and/or from a motor powered by a battery and/or anengine. The automated control system may, for example, operate on thevehicle or on the user, such as to control operation of the vehicle bydetermining whether at a current time to remove energy from the batteryto power the motor (and if so to further determine how much energy toremove from the battery) or to instead add excess energy to the battery(e.g., as generated by the engine, and if so to further determine howmuch energy to generate from the engine; and/or as captured from brakingor downhill coasting). Outputs of such a physical system include theeffects of the motor to move the vehicle, and a goal that the automatedcontrol system implements may be, for example, to move the vehicle atone or more specified speeds with a minimum of energy produced from thebattery, and/or to minimize use of fuel by the engine. It will beappreciated that such a physical system being controlled and acorresponding automated control system may include a variety of elementsand use various types of information and perform various types ofactivities, with additional details regarding such an automated controlsystem being included below.

In one or more embodiments, an automated control system is generated andprovided and used to manage product inventory for one or more productsat one or more locations, such as a retail location that receivesproducts from one or more product sources (e.g., when ordered orrequested by the retail location) and that provides products to one ormore product recipients (e.g., when ordered or requested by therecipients). The automated control system may, for example, operate atthe retail location and/or at a remote network-accessible location, suchas to receive requests from product recipients for products, and tocontrol operation of the product inventory at the one or more locationsby selecting at a current time one or more first amounts of one or moreproducts to request from the one or more product sources, and byselecting at the current time one or more second amounts of at least oneproduct to provide to the one or more product recipients. Outputs ofsuch a physical system include products being provided from the one ormore locations to the one or more product recipients, and a goal thatthe automated control system implements may be, for example, to maximizeprofit of an entity operating the one or more locations whilemaintaining the inventory at one or more specified levels. It will beappreciated that such a physical system being controlled and acorresponding automated control system may include a variety of elementsand use various types of information and perform various types ofactivities, with additional details regarding such an automated controlsystem being included below.

In one or more embodiments, an automated control system is generated andprovided and used to manage cyber-security for physical computingresources being protected from unauthorized operations and/or todetermine a risk level from information provided by or available fromone or more information sources. The automated control system may, forexample, operate at the location of the computing resources orinformation sources and/or at a remote network-accessible location, suchas to receive information about attempts (whether current or past) toperform operations on computing resources being protected or aboutinformation being provided by or available from the one or moreinformation sources, and to control operation of the cyber-securitysystem by determine whether a change in authorization to a specifiedtype of operation is needed and to select one or more actions to take toimplement the change in authorization if so determined, and/or todetermine whether a risk level exceeds a specified threshold and toselect one or more actions to take to mitigate the risk level. A goalthat the automated control system implements may be, for example, tominimize unauthorized operations that are performed and/or to minimizethe risk level. It will be appreciated that such a target system beingcontrolled and a corresponding automated control system may include avariety of elements and use various types of information and performvarious types of activities, with additional details regarding such anautomated control system being included below.

In one or more embodiments, an automated control system is generated andprovided and used to manage transactions being performed in one or morefinancial markets, such as to buy and/or sell physical items or otherfinancial items. The automated control system may, for example, operateat the one or more final markets or at a network-accessible locationthat is remote from the one or more financial markets, such as tocontrol operation of the transactions performed by determining whetherto purchase or sell particular items at particular times and to selectone or more actions to initiate transactions to purchase or sell theparticular items at the particular times. A goal that the automatedcontrol system implements may be, for example, to maximize profit whilemaintaining risk below a specified threshold. It will be appreciatedthat such a target system being controlled and a corresponding automatedcontrol system may include a variety of elements and use various typesof information and perform various types of activities, with additionaldetails regarding such an automated control system being included below.

In one or more embodiments, an automated control system is generated andprovided and used to perform coding for medical procedures, such as toallow billing to occur for medical procedures performed on humans. Theautomated control system may, for example, operate at a location atwhich the medical procedures are performed or at a network-accessiblelocation that is remote from such a medical location, such as to controloperation of the coding that is performed by selecting particularmedical codes to associate with particular medical procedures inspecified circumstances. A goal that the automated control systemimplements may be, for example, to minimize errors in selected medicalcodes that cause revenue leakage. It will be appreciated that such atarget system being controlled and a corresponding automated controlsystem may include a variety of elements and use various types ofinformation and perform various types of activities, with additionaldetails regarding such an automated control system being included below.

It will also be appreciated that the described techniques may be usedwith a wide variety of other types of target systems, some of which arefurther discussed below, and that the invention is not limited to thetechniques discussed for particular target systems and correspondingautomated control systems.

As noted above, a Collaborative Distributed Decision (CDD) system may insome embodiments use at least some of the described techniques toperform various automated activities involved in constructing andimplementing a automated control system for a specified target system,such as to modify or otherwise manipulate inputs or other controlelements of the target system that affect its operation (e.g., affectone or more outputs of the target system). An automated control systemfor such a target system may in some situations have a distributedarchitecture that provides cooperative distributed control of the targetsystem, such as with multiple decision modules that each control aportion of the target system and that operate in a partially decoupledmanner with respect to each other. If so, the various decision modules'operations for the automated control system may be at least partiallysynchronized, such as by each reaching a consensus with one or moreother decision modules at one or more times, even if a fullysynchronized convergence of all decision modules at all times is notguaranteed or achieved.

The CDD system may in some embodiments implement a Decision ModuleConstruction component that interacts with one or more users to obtain adescription of a target system, including restrictions related to thevarious elements of the target system, and one or more goals to beachieved during control of the target system—the Decision ModuleConstruction component then performs various automated actions togenerate, test and deploy one or more executable decision modules to usein performing the control of the target system. The Decision ModuleConstruction component may thus operate as part of a configuration orsetup phase that occurs before a later run-time phase in which thegenerated decision modules are executed to perform control of the targetsystem, although in some embodiments and situations the Decision ModuleConstruction component may be further used after an initial deploymentto improve or extend or otherwise modify an automated control systemthat has one or more decision modules (e.g., while the automated controlsystem continues to be used to control the target system), such as toadd, remove or modify decision modules for the automated control system.

In some embodiments, some or all automated control systems that aregenerated and deployed may further provide various components withinthem for execution during the runtime operation of the automated controlsystem, such as by including such components within decision modules insome embodiments and situations. Such components may include, forexample, a Control Action Determination component of each decisionmodule (or of some decision modules) to optimize or otherwise determineand improve the control actions that the decision module generates. Forexample, such a Control Action Determination component in a decisionmodule may in some embodiments attempt to automatically determine thedecision module's control actions for a particular time to reflect anear-optimal solution with respect to or one more goals and in light ofa model of the decision module for the target system that has multipleinter-related constraints—if so, such a near-optimal solution may bebased at least in part on a partially optimized solution that is withina threshold amount of a fully optimized solution. Such determination ofone or more control actions to perform may occur for a particular timeand for each of one or more decision modules, as well as be repeatedover multiple times for ongoing control by at least some decisionmodules in some situations. In some embodiments, the model for adecision module is implemented as a Hamiltonian function that reflects aset of coupled differential equations based in part on constraintsrepresenting at least part of the target system, such as to allow themodel and its Hamiltonian function implementation to be updated overmultiple time periods by adding additional expressions within theevolving Hamiltonian function.

In some embodiments, the components included within a generated anddeployed automated control system for execution during the automatedcontrol system's runtime operation may further include one or moreCoordinated Control Management components to coordinate the controlactions of multiple decision modules that are collectively performingthe control of a target system for the automated control system. Forexample, some or all decision modules may each include such a ControlAction Determination component in some embodiments to attempt tosynchronize that decision module's local solutions and proposed controlactions with those of one or more other decision modules in theautomated control system, such as by determining a consensus sharedmodel with those other decision modules that simultaneously providessolutions from the decision module's local model and the models of theone or more other decision modules. Such inter-module synchronizationsmay occur repeatedly to determine one or more control actions for eachdecision module at a particular time, as well as to be repeated overmultiple times for ongoing control. In addition, each decision module'smodel is implemented in some embodiments as a Hamiltonian function thatreflects a set of coupled differential equations based in part onconstraints representing at least part of the target system, such as toallow each decision module's model and its Hamiltonian functionimplementation to be combined with the models of one or more otherdecision modules by adding additional expressions for those otherdecision modules' models within the initial Hamiltonian function for thelocal model of the decision module.

Use of the described techniques may also provide various types ofbenefits in particular embodiments, including non-exclusive examples ofbeneficial attributes or operations as follows:

-   -   Infer interests/desired content in a cold start environment        where textual (or other unstructured) data is available and with        minimal user history;    -   Improve inference in a continuous way that can incorporate        increasingly rich user histories;    -   Improve inference performance with the addition of feedback,        explicit/implicit, positive/negative and preferably in a        real-time or near-real-time manner;    -   Derive information from domain experts that provide business        value, and embed them in inference framework;    -   Dynamically add new unstructured data that may represent new        states, and update existing model in a calibrated way;    -   Renormalize inference system to accommodate conflicts;    -   Immediately do inferencing in a new environment based on a        natural language model;    -   Add new information as a statistical model, and integrate with a        natural language model to significantly improve        inference/prediction;    -   Integrate new data and disintegrate old data in a way that only        improves performance;    -   Perform inferencing in a data secure way;    -   Integrate distinct inferencing elements in a distributed network        and improve overall performance;    -   Easily program rules and information into the system from a        lay-user perspective;    -   Inexpensively perform computer inferences in a way that is        suitable for bandwidth of mobile devices; and    -   Incorporate constraint information.

It will be appreciated that some embodiments may not include all someillustrative benefits, and that some embodiments may include somebenefits that are not listed.

For illustrative purposes, some embodiments are described below in whichspecific types of operations are performed, including with respect tousing the described techniques with particular types of target systemsand to perform particular types of control activities that aredetermined in particular manners. These examples are provided forillustrative purposes and are simplified for the sake of brevity, andthe inventive techniques may be used in a wide variety of othersituations, including in other environments and with other types ofautomated control action determination techniques, some of which arediscussed below.

FIG. 1 is a network diagram illustrating an example environment in whicha system for performing cooperative distributed control of one or moretarget systems may be configured and initiated. In particular, anembodiment of a CDD system 140 is executing on one or more computingsystems 190, including in the illustrated embodiment to operate in anonline manner and provide a graphical user interface (GUI) (not shown)and/or other interfaces 119 to enable one or more remote users of clientcomputing systems 110 to interact over one or more intervening computernetworks 100 with the CDD system 140 to configure and create one or moredecision modules to include as part of an automated control system touse with each of one or more target systems to be controlled.

In particular, target system 1 160 and target system 2 170 are exampletarget systems illustrated in this example, although it will beappreciated that only one target system or numerous target systems maybe available in particular embodiments and situations, and that eachsuch target system may include a variety of mechanical, electronic,chemical, biological, and/or other types of components to implementoperations of the target system in a manner specific to the targetsystem. In this example, the one or more users (not shown) may interactwith the CDD system 140 to generate an example automated control system122 for target system 1, with the automated control system includingmultiple decision modules 124 in this example that will cooperativelyinteract to control portions of the target system 1 160 when laterdeployed and implemented. The process of the users interacting with theCDD system 140 to create the automated control system 122 may involve avariety of interactions over time, including in some cases independentactions of different groups of users, as discussed in greater detailelsewhere. In addition, as part of the process of creating and/ortraining or testing automated control system 122, it may perform one ormore interactions with the target system 1 as illustrated, such as toobtain partial initial state information, although some or all trainingactivities may in at least some embodiments include simulating effectsof control actions in the target system 1 without actually implementingthose control actions at that time.

After the automated control system 122 is created, the automated controlsystem may be deployed and implemented to begin performing operationsinvolving controlling the target system 1 160, such as by optionallyexecuting the automated control system 122 on the one or more computingsystems 190 of the CDD system 140, so as to interact over the computernetworks 100 with the target system 1. In other embodiments andsituations, the automated control system 122 may instead be deployed byexecuting local copies of some or all of the automated control system122 (e.g., one or more of the multiple decision modules 124) in a mannerlocal to the target system 1, as illustrated with respect to a deployedcopy 121 of some or all of automated control system 1, such as on one ormore computing systems (not shown) that are part of the target system 1.

In a similar manner to that discussed with respect to automated controlsystem 122, one or more users (whether the same users, overlappingusers, or completely unrelated users to those that were involved increating the automated control system 122) may similarly interact overthe computer network 100 with the CDD system 140 to create a separateautomated control system 126 for use in controlling some or all of thetarget system 2 170. In this example, the automated control system 126for target system 2 includes only a single decision module 128 that willperform all of the control actions for the automated control system 126.The automated control system 126 may similarly be deployed andimplemented for target system 2 in a manner similar to that discussedwith respect to automated control system 122, such as to execute locallyon the one or more computing systems 190 and/or on one or more computingsystems (not shown) that are part of the target system 2, although adeployed copy of automated control system 2 is not illustrated in thisexample. It will be further appreciated that the automated controlsystems 122 and/or 126 may further include other components and/orfunctionality that are separate from the particular decision modules 124and 128, respectively, although such other components and/orfunctionality are not illustrated in FIG. 1.

The network 100 may, for example, be a publicly accessible network oflinked networks, possibly operated by various distinct parties, such asthe Internet, with the CDD system 140 available to any users or onlycertain users over the network 100. In other embodiments, the network100 may be a private network, such as, for example, a corporate oruniversity network that is wholly or partially inaccessible tonon-privileged users. In still other embodiments, the network 100 mayinclude one or more private networks with access to and/or from theInternet. Thus, while the CDD system 140 in the illustrated embodimentis implemented in an online manner to support various users over the oneor more computer networks 100, in other embodiments a copy of the CDDsystem 140 may instead be implemented in other manners, such as tosupport a single user or a group of related users (e.g., a company orother organization), such as if the one or more computer networks 100are instead an internal computer network of the company or otherorganization, and with such a copy of the CDD system optionally notbeing available to other users external to the company or otherorganizations. The online version of the CDD system 140 and/or localcopy version of the CDD system 140 may in some embodiments andsituations operate in a fee-based manner, such that the one or moreusers provide various fees to use various operations of the CDD system,such as to perform interactions to generate decision modules andcorresponding automated control systems, and/or to deploy or implementsuch decision modules and corresponding automated control systems invarious manners. In addition, the CDD system 140, each of its components(including component 142 and optional other components 117, such as oneor more CDD Control Action Determination components and/or one or moreCDD Coordinated Control Management components), each of the decisionmodules, and/or each of the automated control systems may includesoftware instructions that execute on one or more computing systems (notshown) by one or more processors (not shown), such as to configure thoseprocessors and computing systems to operate as specialized machines withrespect to performing their programmed functionality.

FIG. 2 is a network diagram illustrating an example environment in whicha system for performing cooperative distributed control of targetsystems may be implemented, and in particular continues the examplesdiscussed with respect to FIG. 1. In the example environment of FIG. 2,target system 1 160 is again illustrated, with the automated controlsystem 122 now being deployed and implemented to use in activelycontrolling the target system 1 160. In the example of FIG. 2, thedecision modules 124 are represented as individual decision modules 124a, 124 b, etc., to 124 n, and may be executing locally to the targetsystem 1 160 and/or in a remote manner over one or more interveningcomputer networks (not shown). In the illustrated example, each of thedecision modules 124 includes a local copy of a CDD Control ActionDetermination component 144, such as with component 144 a supporting itslocal decision module 124 a, component 144 b supporting its localdecision module 124 b, and component 144 n supporting its local decisionmodule 124 n. Similarly, the actions of the various decision modules 124are coordinated and synchronized in a peer-to-peer manner in theillustrated embodiment, with each of the decision modules 124 includinga copy of a CDD Coordinated Control Management component 146 to performsuch synchronization, with component 146 a supporting its local decisionmodule 124 a, component 146 b supporting its local decision module 124b, and component 146 n supporting its local decision module 124 n.

As the decision modules 124 and automated control system 122 execute,various interactions 175 between the decision modules 124 are performed,such as to share information about current models and other state of thedecision modules to enable cooperation and coordination between variousdecision modules, such as for a particular decision module to operate ina partially synchronized consensus manner with respect to one or moreother decision modules (and in some situations in a fully synchronizedmanner in which the consensus actions of all of the decision modules 124converge). During operation of the decision modules 124 and automatedcontrol system 122, various state information 143 may be obtained by theautomated control system 122 from the target system 160, such as initialstate information and changing state information over time, andincluding outputs or other results in the target system 1 from controlactions performed by the decision modules 124.

The target system 1 in this example includes various control elements161 that the automated control system 122 may manipulate, and in thisexample each decision module 124 may have a separate group of one ormore control elements 161 that it manipulates (such that decision moduleA 124 a performs interactions 169 a to perform control actions A 147 aon control elements A 161 a, decision module B 124 b performsinteractions 169 b to perform control actions B 147 b on controlelements B 161 b, and decision module N 124 n performs interactions 169n to perform control actions N 147 n on control elements N 161 n). Suchcontrol actions affect the internal state 163 of other elements of thetarget system 1, including optionally to cause or influence one or moreoutputs 162. As operation of the target system 1 is ongoing, at leastsome of the internal state information 163 is provided to some or all ofthe decision modules to influence their ongoing control actions, witheach of the decision modules 124 a-124 n possibly having a distinct setof state information 143 a-143 n, respectively, in this example.

As discussed in greater detail elsewhere, each decision module 124 mayuse such state information 143 and a local model 145 of the decisionmodule for the target system to determine particular control actions 147to next perform, such as for each of multiple time periods, although inother embodiments and situations, a particular automated control systemmay perform interactions with a particular target system for only onetime period or only for some time periods. For example, the local CDDControl Action Determination component 144 for a decision module 124 maydetermine a near-optimal location solution for that decision module'slocal model 145, and with the local CDD Coordinated Control Managementcomponent 146 determining a synchronized consensus solution to reflectother of the decision modules 124, including to update the decisionmodule's local model 145 based on such local and/or synchronizedsolutions that are determined. Thus, during execution of the automatedcontrol system 122, the automated control system performs variousinteractions with the target system 160, including to request stateinformation, and to provide instructions to modify values of orotherwise manipulate control elements 161 of the target system 160. Forexample, for each of multiple time periods, decision module 124 a mayperform one or more interactions 169 a with one or more control elements161 a of the target system, while decision module 124 b may similarlyperform one or more interactions 169 b with one or more separate controlelements B 161 b, and decision module 124 n may perform one or moreinteractions 169 n with one or more control elements N 161 n of thetarget system 160. In other embodiments and situations, at least somecontrol elements may not perform control actions during each timeperiod.

While example target system 2 170 is not illustrated in FIG. 2, furtherdetails are illustrated for decision module 128 of automated controlsystem 126 for reference purposes, although such a decision module 128would not typically be implemented together with the decision modules124 controlling target system 1. In particular, the deployed copy ofautomated control system 126 includes only the single executing decisionmodule 128 in this example, although in other embodiments the automatedcontrol system 126 may include other components and functionality. Inaddition, since only a single decision module 128 is implemented for theautomated control system 126, the decision module 128 includes a localCDD Control Action Determination component 244, but does not in theillustrated embodiment include any local CDD Coordinated ControlManagement component, since there are not other decision modules withwhich to synchronize and interact.

While not illustrated in FIGS. 1 and 2, the distributed nature ofoperations of automated control systems such as those of 122 allowpartially decoupled operations of the various decision modules, includeto allow modifications to the group of decision modules 124 to bemodified over time while the automated control system 122 is in use,such as to add new decision modules 124 and/or to remove existingdecision modules 124. In a similar manner, changes may be made toparticular decision modules 124 and/or 128, such as to change rules orother restrictions specific to a particular decision module and/or tochange goals specific to a particular decision module over time, with anew corresponding model being generated and deployed within such adecision module, including in some embodiments and situations while thecorresponding automated control system continues control operations of acorresponding target system. In addition, while each automated controlsystem is described as controlling a single target system in theexamples of FIGS. 1 and 2, in other embodiments and situations, otherconfigurations may be used, such as for a single automated controlsystem to control multiple target systems (e.g., multiple inter-relatedtarget systems, multiple target systems of the same type, etc.), and/ormultiple automated control systems may operate to control a singletarget system, such as by each operating independently to controldifferent portions of that target control system. It will be appreciatedthat other configurations may similarly be used in other embodiments andsituations.

FIG. 3 is a block diagram illustrating example computing systemssuitable for performing techniques for implementing automated controlsystems to control or otherwise manipulate at least some operations ofspecified physical systems or other target systems in configuredmanners. In particular, FIG. 3 illustrates a server computing system 300suitable for providing at least some functionality of a CDD system,although in other embodiments multiple computing systems may be used forthe execution (e.g., to have distinct computing systems executing theCDD Decision Module Construction component for initial configuration andsetup before run-time control occurs, and one or more copies of the CDDControl Action Determination component 344 and/or the CDD CoordinatedControl Managements component 346 for the actual run-time control). FIG.3 also illustrates various client computer systems 350 that may be usedby customers or other users of the CDD system 340, as well as one ormore target systems (in this example, target system 1 360 and targetsystem 2 370, which are accessible to the CDD system 340 over one ormore computer networks 390).

The server computing system 300 has components in the illustratedembodiment that include one or more hardware CPU (“central processingunit”) computer processors 305, various I/O (“input/output”) hardwarecomponents 310, storage 320, and memory 330. The illustrated I/Ocomponents include a display 311, a network connection 312, acomputer-readable media drive 313, and other I/O devices 315 (e.g., akeyboard, a mouse, speakers, etc.). In addition, the illustrated clientcomputer systems 350 may each have components similar to those of servercomputing system 300, including one or more CPUs 351, I/O components352, storage 354, and memory 357, although some details are notillustrated for the computing systems 350 for the sake of brevity. Thetarget systems 360 and 370 may also each include one or more computingsystems (not shown) having components that are similar to some or all ofthe components illustrated with respect to server computing system 300,but such computing systems and components are not illustrated in thisexample for the sake of brevity.

The CDD system 340 is executing in memory 330 and includes components342-346, and in some embodiments the system and/or components eachincludes various software instructions that when executed program one ormore of the CPU processors 305 to provide an embodiment of a CDD systemas described elsewhere herein. The CDD system 340 may interact withcomputing systems 350 over the network 390 (e.g., via the Internetand/or the World Wide Web, via a private cellular network, etc.), aswell as the target systems 360 and 370 in this example. In this exampleembodiment, the CDD system includes functionality related to generatingand deploying decision modules in configured manners for customers orother users, as discussed in greater detail elsewhere herein. The othercomputing systems 350 may also be executing various software as part ofinteractions with the CDD system 340 and/or its components. For example,client computing systems 350 may be executing software in memory 357 tointeract with CDD system 340 (e.g., as part of a Web browser, aspecialized client-side application program, etc.), such as to interactwith one or more interfaces (not shown) of the CDD system 340 toconfigure and deploy automated control systems (e.g., stored automatedcontrol systems 325 that were previously created by the CDD system 340)or other decision modules 329, as well as to perform various other typesof actions, as discussed in greater detail elsewhere. Variousinformation related to the functionality of the CDD system 340 may bestored in storage 320, such as information 321 related to users of theCDD system (e.g., account information), and information 323 related toone or more target systems. It will be appreciated that computingsystems 300 and 350 and target systems 360 and 370 are merelyillustrative and are not intended to limit the scope of the presentinvention. The computing systems may instead each include multipleinteracting computing systems or devices, and the computingsystems/nodes may be connected to other devices that are notillustrated, including through one or more networks such as theInternet, via the Web, or via private networks (e.g., mobilecommunication networks, etc.). More generally, a computing node or othercomputing system or device may comprise any combination of hardware thatmay interact and perform the described types of functionality, includingwithout limitation desktop or other computers, database servers, networkstorage devices and other network devices, PDAs, cell phones, wirelessphones, pagers, electronic organizers, Internet appliances,television-based systems (e.g., using set-top boxes and/orpersonal/digital video recorders), and various other consumer productsthat include appropriate communication capabilities. In addition, thefunctionality provided by the illustrated CDD system 340 and itscomponents may in some embodiments be distributed in additionalcomponents. Similarly, in some embodiments some of the functionality ofthe CDD system 340 and/or CDD components 342-346 may not be providedand/or other additional functionality may be available.

It will also be appreciated that, while various items are illustrated asbeing stored in memory or on storage while being used, these items orportions of them may be transferred between memory and other storagedevices for purposes of memory management and data integrity.Alternatively, in other embodiments some or all of the software modulesand/or systems may execute in memory on another device and communicatewith the illustrated computing systems via inter-computer communication.Thus, in some embodiments, some or all of the described techniques maybe performed by hardware means that include one or more processorsand/or memory and/or storage when configured by one or more softwareprograms (e.g., by the CDD system 340 and/or the CDD components 342-346)and/or data structures, such as by execution of software instructions ofthe one or more software programs and/or by storage of such softwareinstructions and/or data structures. Furthermore, in some embodiments,some or all of the systems and/or components may be implemented orprovided in other manners, such as by using means that are implementedat least partially or completely in firmware and/or hardware, including,but not limited to, one or more application-specific integrated circuits(ASICs), standard integrated circuits, controllers (e.g., by executingappropriate instructions, and including microcontrollers and/or embeddedcontrollers), field-programmable gate arrays (FPGAs), complexprogrammable logic devices (CPLDs), etc. Some or all of the components,systems and data structures may also be stored (e.g., as softwareinstructions or structured data) on a non-transitory computer-readablestorage medium, such as a hard disk or flash drive or other non-volatilestorage device, volatile or non-volatile memory (e.g., RAM), a networkstorage device, or a portable media article to be read by an appropriatedrive (e.g., a DVD disk, a CD disk, an optical disk, etc.) or via anappropriate connection. The systems, components and data structures mayalso in some embodiments be transmitted as generated data signals (e.g.,as part of a carrier wave or other analog or digital propagated signal)on a variety of computer-readable transmission mediums, includingwireless-based and wired/cable-based mediums, and may take a variety offorms (e.g., as part of a single or multiplexed analog signal, or asmultiple discrete digital packets or frames). Such computer programproducts may also take other forms in other embodiments. Accordingly,the present invention may be practiced with other computer systemconfigurations.

FIG. 4 is a flow diagram of an example embodiment of a CollaborativeDistributed Decision (CDD) system routine 400. The routine may, forexample, be provided by execution of the CDD system 340 of FIG. 3 and/orthe CDD system 140 of FIG. 1, such as to provide functionality toconstruct and implement automated control systems for specified targetsystems.

The illustrated embodiment of the routine begins at block 410, whereinformation or instructions are received. If it is determined in block420 that the information or instructions of block 410 include anindication to create or revise one or more decision modules for use aspart of an automated control system for a particular target system, theroutine continues to block 425 to initiate execution of a DecisionModule Construction component, and in block 430 obtains and stores oneor more resulting decision modules for the target system that arecreated in block 425. One example of a routine for such a DecisionModule Construction component is discussed in greater detail withrespect to FIGS. 5A-5B.

After block 430, or if it is instead determined in block 420 that theinformation or instructions received in block 410 are not to create orrevise one or more decision modules, the routine continues to block 440to determine whether the information or instructions received in block410 indicate to deploy one or more created decision modules to control aspecified target system, such as for one or more decision modules thatare part of an automated control system for that target system. The oneor more decision modules to deploy may have been created immediatelyprior with respect to block 425, such that the deployment occurs in amanner that is substantially simultaneous with the creation, or in othersituations may include one or more decision modules that were created ata previous time and stored for later use. If it is determined to deployone or more such decision modules for such a target system, the routinecontinues to block 450 to initiate the execution of those one or moredecision modules for that target system, such as on one or morecomputing systems local to an environment of the target system, orinstead on one or more remote computing systems that communicate withthe target system over one or more intermediary computer networks (e.g.,one or more computing systems under control of a provider of the CDDsystem).

After block 450, the routine continues to block 460 to determine whetherto perform distributed management of multiple decision modules beingdeployed in a manner external to those decision modules, such as via oneor more centralized Coordinated Control Management components. If so,the routine continues to block 465 to initiate execution of one or moresuch centralized CDD Coordinated Control Management components for usewith those decision modules. After block 465, or if it is insteaddetermined in block 460 to not perform such distributed management in anexternal manner (e.g., if only one decision module is executed, ifmultiple decision modules are executed but coordinate their operationsin a distributed peer-to-peer manner, etc.), the routine continues toblock 470 to optionally obtain and store information about theoperations of the one or more decision modules and/or resultingactivities that occur in the target system, such as for later analysisand/or reporting.

If it is instead determined in block 440 that the information orinstructions received in block 410 are not to deploy one or moredecision modules, the routine continues instead to block 485 to performone or more other indicated operations if appropriate. For example, suchother authorized operations may include obtaining results informationabout the operation of a target system in other manners (e.g., bymonitoring outputs or other state information for the target system),analyzing results of operations of decision modules and/or activities ofcorresponding target systems, generating reports or otherwise providinginformation to users regarding such operations and/or activities, etc.In addition, in some embodiments the analysis of activities of aparticular target system over time may allow patterns to be identifiedin operation of the target system, such as to allow a model of thattarget system to be modified accordingly (whether manually or in anautomated learning manner) to reflect those patterns and to respondbased on them. In addition, as discussed in greater detail elsewhere,distributed operation of multiple decision modules for an automatedcontrol system in a partially decoupled manner allows various changes tobe made while the automated control system is in operation, such as toadd one or more new decision modules, to remove one or more existingdecision modules, to modify the operation of a particular decisionmodule (e.g., by changing rules or other information describing thetarget system that is part of a model for the decision module), etc. Inaddition, the partially decoupled nature of multiple such decisionmodules in an automated control system allows one or more such decisionmodules to operate individually at times, such as if networkcommunication issues or other problems prevent communication betweenmultiple decision modules that would otherwise allow theirindividualized control actions to be coordinated—in such situations,some or all such decision modules may continue to operate in anindividualized manner, such as to provide useful ongoing controloperations for a target system even if optimal or near-optimal solutionscannot be identified from coordination and synchronization between agroup of multiple decision modules that collectively provide theautomated control system for the target system.

After blocks 470 or 485, the routine continues to block 495 to determinewhether to continue, such as until an explicit indication to terminateis received. If it is determined to continue, the routine returns toblock 410, and otherwise continues to block 499 and ends.

FIGS. 5A-5B illustrate a flow diagram of an example embodiment of a CDDDecision Module Construction routine 500. The routine may, for example,be provided by execution of the component 342 of FIG. 3 and/or thecomponent 142 of FIG. 1, such as to provide functionality to allow usersto provide information describing a target system of interest, and toperform corresponding automated operations to construct one or moredecision modules to use to control the target system in specifiedmanners. While the illustrated embodiment of the routine interacts withusers in particular manners, such as via a displayed GUI (graphical userinterface), it will be appreciated that other embodiments of the routinemay interact with users in other manners, such as via a defined API(application programming interface) that an executing program invokes onbehalf of a user. In some embodiments, the routine may further beimplemented as part of an integrated development environment or othersoftware tool that is available for one or more users to use, such as byimplementing an online interface that is available to a variety ofremote users over a public network such as the Internet, while in otherembodiments a copy of the CDD system and/or particular CDD componentsmay be used to support a single organization or other group of one ormore users, such as by being executed on computing systems under thecontrol of the organization or group. In addition, the CDD DecisionModule Construction component may in some embodiments and situations beseparated into multiple sub-components, such as a rules editor componentthat users interact with to specify rules and other descriptioninformation for a target system, and a rules compiler engine thatprocesses the user-specified rules and other information to create oneor more corresponding decision modules.

The illustrated embodiment of the routine 500 begins at block 510, wherethe routine provides or updates a displayed user interface to one ormore users, such as via a request received at an online version ofcomponent that is implementing the routine, or instead based on theroutine being executed by one or more such users on computing systemsthat they control. While various operations are shown in the illustratedembodiment of the routine as occurring in a serial manner for thepurpose of illustration, it will be appreciated that user interactionswith such a user interface may occur in an iterative manner and/or overmultiple periods of time and/or user sessions, including to update auser interface previously displayed to a user in various manners (e.g.,to reflect a user action, to reflect user feedback generated byoperation of the routine or from another component, etc.), as discussedfurther below.

After block 510, the routine continues to block 520 to receiveinformation from one or more such users describing a target system to becontrolled, including information about a plurality of elements of thetarget system that include one or more manipulatable control elementsand optionally one or more outputs that the control elements affect,information about rules that specify restrictions involving theelements, information about state information that will be availableduring controlling of the system (e.g., values of particular elements orother state variables), and one or more goals to achieve during thecontrolling of the target system. It will be appreciated that suchinformation may be obtained over a period of time from one or moreusers, including in some embodiments for a first group of one or moreusers to supply some information related to a target system and for oneor more other second groups of users to independently provide otherinformation about the target system, such as to reflect different areasof expertise of the different users and/or different parts of the targetsystem.

After block 520, the routine continues to block 525 to identify anyerrors that have been received in the user input, and to prompt theuser(s) to correct those errors, such as by updating the display in acorresponding manner as discussed with respect to block 510. While theidentification of such errors is illustrated as occurring after thereceiving of the information in block 520, it will be appreciated thatsome or all such errors may instead be identified as the users areinputting information into the user interface, such as to identifysyntax errors in rules or other information that the users specify.After block 525, the illustrated embodiment of the routine continues toblock 530 to optionally decompose the information about the targetsystem into multiple subsets that each correspond to a portion of thetarget system, such as with each subset having one or more differentcontrol elements that are manipulatable by the automated control systembeing created by the routine, and optionally have overlapping orcompletely distinct goals and/or sets of rules and other informationdescribing the respective portions of the target system. As discussed ingreater detail elsewhere, such decomposition, if performed, may in somesituations be performed manually by the users indicating differentsubgroups of information that they enter, and/or in an automated mannerby the routine based on an analysis of the information that has beenspecified (e.g., based on the size of rules and other descriptiveinformation supplied for a target system, based on inter-relationshipsbetween different rules or goals or other information, etc.). In otherembodiments, no such decomposition may be performed.

After block 530, the routine continues to block 535 to, for each subsetof target system description information (or for all the receivedinformation if no such subsets are identified), convert that subset (orall the information) into a set of constraints that encapsulate therestrictions, goals, and other specified information for that subset (orfor all the information). In block 540, the routine then identifies anyerrors that occur from the converting process, and if any areidentified, may prompt the user to correct those errors, such as in amanner similar to that described with respect to blocks 525 and 510.While not illustrated in this example, the routine may in somesituations in blocks 525 and/or 540 return to block 510 when such errorsare identified, to display corresponding feedback to the user(s) and toallow the user(s) to make corrections and re-perform followingoperations such as those of blocks 520-540. The errors identified in theconverting process in block 540 may include, for example, errors relatedto inconsistent restrictions, such as if the restrictions as a group areimpossible to satisfy.

After block 540, the routine continues to block 545 to, for each set ofconstraints (or a single constraint set if no subsets were identified inblock 530), apply one or more validation rules to the set of constraintsto test overall effectiveness of the corresponding information that theconstraints represent, and to prompt the one or more users to correctany errors that are identified in a manner similar to that with respectto blocks 525, 540 and 510. Such validation rules may test one or moreof controllability, observability, stability, and goal completeness, aswell as any user-added validation rules, as discussed in greater detailelsewhere. In block 550, the routine then converts each validated set ofconstraints to a set of coupled differential equations that model atleast a portion of the target system to which the underlying informationcorresponds.

After block 550, the routine continues to block 553 to performactivities related to training a model for each set of coupleddifferential equations, including to determine one or more of a size ofa training time window to use, size of multiple training time sliceswithin the time window, and/or a type of training time slice within thetime window. In some embodiments and situations, the determination ofone or more such sizes or types of information is performed by usingdefault or pre-specified information, while in other embodiments andsituations the users may specify such information, or an automateddetermination of such information may be performed in one or moremanners (e.g., by testing different sizes and evaluating results to findsizes with the best performance). Different types of time slices mayinclude, for example, successions of time slices that overlap or do notoverlap, such that the training for a second time slice may be dependentonly on results of a first time slice (if they do not overlap) orinstead may be based at least in part on updating information alreadydetermined for at least some of the first time slice (if they do overlapin part or in whole). After block 553, the routine continues to block555 to, for each set of coupled differential equations representing amodel, train the model for that set of coupled differential equationsusing partial initial state information for the target system, includingto estimate values of variable that are not known and/or directlyobservable for the target system by simulating effects of performingcontrol actions over the time window, such as for successive time slicesthroughout the time window, and to test the simulated performance of thetrained model. Additional details related to training and testing areincluded elsewhere herein.

After block 555, the routine continues to block 560 to determine whetherthe training and testing was successful, and if not returns to block 510to display corresponding feedback information to the users to allow themto correct errors that caused the lack of success. If it is insteaddetermined in block 560 that the testing and training were successful,however, the routine continues instead to block 570 to generate anexecutable decision module for each trained and tested model thatincludes that model, as well as a local CCD Control Action Determinationcomponent that the decision module will use when executed to determineoptimal or near-optimal control actions to perform for the target systembased on the information included in the model, and in light of the oneor more goals for that decision module. The generated executabledecision module may in some embodiments and situations further include alocal CCD Coordinated Control Management component to coordinate controlactions of multiple decision modules that collectively will provide anautomated control system for the target system, such as by synchronizingrespective models of the various decision modules over time. After block570, the routine continues to block 580 to provide the generatedexecutable decision modules for use, including to optionally store themfor later execution and/or deployment.

After block 580, the routine continues to block 595 to determine whetherto continue, such as until an explicit indication to terminate isreceived. If it is determined to continue, the routine returns to block510, and otherwise continues to block 599 and ends.

FIGS. 6A-6B illustrate a flow diagram of an example embodiment of aroutine 600 corresponding to a generic representation of a decisionmodule that is being executed. The routine may, for example, be providedby execution of a decision module 329 or as part of an automated controlsystem 325 of FIG. 3 and/or a decision module 124 or 128 of FIG. 1 or 2,such as to provide functionality for controlling at least a portion of atarget system in a manner specific to information and a model encodedfor the decision module, including to reflect one or more goals to beachieved by the decision module during its controlling activities. Asdiscussed in greater detail elsewhere, in some embodiments andsituations, multiple decision modules may collectively and cooperativelyact to control a particular target system, such as with each decisionmodule controlling one or more distinct control elements for the targetsystem or otherwise representing or interacting with a portion of thetarget system, while in other embodiments and situations a singledecision module may act alone to control a target system. The routine600 further reflects actions performed by a particular example decisionmodule when it is deployed in controlling a portion of a target system,although execution of at least portions of a decision module may occurat other times, such as initially to train a model for the decisionmodule before the decision module is deployed, as discussed in greaterdetail with respect to the CDD Decision Module Construction routine 500of FIGS. 5A-5B.

The illustrated embodiment of the routine 600 begins at block 610, wherean initial model for the decision module is determined that describes atleast a portion of a target system to be controlled, one or more goalsfor the decision module to attempt to achieve related to control of thetarget system, and optionally initial state information for the targetsystem. The routine continues to block 615 to perform one or moreactions to train the initial model if needed, as discussed in greaterdetail with respect to blocks 553 and 555 of FIGS. 5A-5B—in someembodiments and situations, such training for block 615 is performedonly if initial training is not done by the routine 500 of FIGS. 5A-5B,while in other embodiments and situations the training of block 615 isperformed to capture information about a current state of the targetsystem at a time that the decision module begins to execute (e.g., ifnot immediately deployed after initial creation and training) and/or tore-train the model at times as discussed in greater detail with respectto routine 700 of FIGS. 7A-7B as initiated by block 630.

After block 615, the routine continues to block 617 to determine a timeperiod to use for performing each control action decision for thedecision module, such as to reflect a rate at which control elementmodifications in the target system are needed and/or to reflect a rateat which new incoming state information is received that may alterfuture manipulations of the control elements. The routine then continuesto block 620 to start the next time period, beginning with a first timeperiod moving forward from the startup of the execution of the decisionmodule. Blocks 620-680 are then performed in a loop for each such timeperiod going forward until execution of the decision module is suspendedor terminated, although in other embodiments a particular decisionmodule may execute for only a single time period each time that it isexecuted.

In block 625, the routine optionally obtains state information for thetime period, such as current state information that has been receivedfor the target system or one or more related external sources since thelast time period began, and/or by actively retrieving current values ofone or more elements of the target system or corresponding variables asneeded. In block 630, the routine then initiates execution of a localCCD Control Action Determination component of the decision module, withone example of such a routine discussed in greater detail with respectto routine 700 of FIGS. 7A-7B. In block 635, the results of theexecution of the component in block 630 are received, including toeither obtain an updated model for the decision module with a localsolution for the current time period and decision module that includesone or more proposed control action determinations that the decisionmodule may perform for the current time period, or to receive anindication that no local solution was found for the decision module inthe allowed time for the execution of the component in block 630. It isthen determined in block 640 whether a solution was found, and if socontinues to block 642 to store the updated model for the decisionmodule, and otherwise continues to block 643 to use the prior model forthe decision module to determine one or more control actiondeterminations that are proposed for the current time period based on aprevious model (e.g., that does not reflect recent changes in stateinformation and/or recent changes in activities of other decisionmodules, if any), as discussed in greater detail with respect to routine700 of FIGS. 7A-7B.

After blocks 642 or 643, the routine continues to block 644 to determineif other decision modules are collectively controlling portions of thecurrent target system, such as part of the same automated control systemas the local decision module, and if so continues to block 645.Otherwise, the routine selects the local proposed control actions of thedecision module as a final determined control action to perform, andcontinues to block 675 to implement those control actions for thecurrent time period.

If there are other operating decision modules, the routine in block 645determines if the local decision module includes a local copy of a CDDCoordinated Control Management (CCM) component for use in synchronizingthe proposed control action determinations for the decision module'slocal solutions with activities of other decision modules that arecollectively controlling the same target system. If so, the routinecontinues to block 647 to provide the one or more proposed controlaction determinations of the decision module and the correspondingcurrent local model for the decision module to the local CDD CCMcomponent, and otherwise continues to block 649 to provide the one ormore proposed control action determinations for the decision module andthe corresponding local model of the decision module to one or morecentralized CDD CCM components.

After blocks 647 or 649, the routine continues to block 655 to obtainresults of the actions of the CDD CCM component(s) in blocks 647 or 649,including to either obtain a further updated model resulting fromsynchronization of the local model for the current decision module withinformation from one or more other decision modules, such that thefurther updated model indicates one or more final control actiondeterminations to perform for the time period for the current decisionmodule, or an indication that no such synchronization was completed inthe allowed time. The routine continues to block 660 to determinewhether the synchronization was completed, and if so continues to block665 to store the further updated model from the synchronization, andotherwise continues to block 670 to use the prior proposed controlaction determinations locally to the decision module as the finalcontrol action determinations for the time period.

After blocks 665 or 670, the routine continues to block 675 to implementthe one or more final determined control actions for the decision modulein the target system, such as by interacting with one or moreeffectuators in the target system that modify values or otherwisemanipulate one or more control elements of the target system, or byotherwise providing input to the target system to cause suchmodifications or other manipulations to occur. In block 680, the routineoptionally obtains information about the results in the target system ofthe control actions performed, and stores and/or provides information tothe CDD system about such obtained results and/or about the activitiesof the decision module for the current time period.

After block 680, the routine continues to block 695 to determine whetherto continue, such as until an indication to terminate or suspend isreceived (e.g., to reflect an end to current operation of the targetsystem or an end of use of the decision module to control at least aportion of the target system). If it is determined to continue, theroutine returns to block 620 to start the next time period, andotherwise continues to block 699 and ends.

FIGS. 7A-7B are a flow diagram of a example embodiment of a CDD ControlAction Determination routine 700. The routine may, for example, beprovided by execution of the component 344 of FIG. 3 and/or components144 a-n or 244 of FIG. 2, such as to determine control actions for adecision module to propose and/or implement for a target system during aparticular time period, including in some embodiments to perform anoptimization to determine near-optimal actions (e.g., within a thresholdof an optimal solution) to perform with respect to one or more goals ifpossible. While the illustrated embodiment of the routine is performedin a manner local to a particular decision module, such that some or alldecision modules may each implement a local version of such a routine,in other embodiments the routine may be implemented in a centralizedmanner by one or more components with which one or more decision modulesinteract over one or more networks, such as with a particular decisionmodule indicated to be used at a particular time rather than acting onbehalf of the local decision module.

The illustrated embodiment of the routine 700 begins at block 703, whereinformation or a request is received. The routine continues to block 705to determine a type of the information or request, and to proceedaccordingly. In particular, if a request is received in block 703 toattempt to determine a solution for a current time period given acurrent model of the local decision module, the routine continues toblock 710 to begin to perform such activities, as discussed in greaterdetail with respect to block 710-790. If it is instead determined inblock 705 that a request to relax one or more rules or otherrestrictions for the current model of the local decision module isreceived, such as discussed in greater detail with respect to blocks 760and 765, the routine continues to block 765. If it is determined inblock 705 that a request is received to repair one or more rules orother restrictions for the current model of the local decision module,such as discussed in greater detail with respect to blocks 775 and 780,the routine continues to block 780 to obtain user input to use duringthe rule repair process (e.g., to interact with a CDD Decision ModuleConstruction component, or to instead interact with one or more users inanother manner), such as to allow the current model for the localdecision module to later be updated and replaced based on furtherresulting user actions, or if operation of the target system can besuspended, to optionally wait to further perform the routine 700 untilsuch an updated model is received. If it is instead determined in block705 that the information or request is of another type, the routinecontinues instead to block 708 to perform one or more other indicatedoperations as appropriate, and to then proceed to block 799. Such otherindicated operations may include, for example, receiving informationabout current models and/or control actions proposed or performed by oneor more other decision modules that are collectively controlling atarget system with the local decision module (such as for use insynchronizing the model of the local decision module with such otherdecision modules by generating a consensus or converged shared model, asdiscussed in greater detail with respect to routine 800 of FIGS. 8A-8B),to receive updates to a model or underlying information for the modelfor use in ongoing operation of the routine 700 (e.g., from a CDDDecision Module Construction component, such as results frominteractions performed in block 780), to receive current stateinformation for the target system, such as for use as discussed inroutine 600 of FIGS. 6A-6B, etc.

If it determined in block 705 that a request for a solution was receivedin block 703 for a current time period and based on a current model ofthe local decision module, the routine continues to block 710 to receivea current set of coupled differential equations that represent thecurrent model for the local decision module of at least a portion of thetarget system, optionally along with additional state information forthe target system for the current time. The routine then continues toblock 715 to determine whether to train or re-train the model, such asif the routine is called for a first time upon initial execution of acorresponding decision module or if error measurements from ongoingoperations indicate a need for re-training, as discussed in greaterdetail with respect to blocks 755, 770 and 730. If it is determined totrain or re-train the model, the routine continues to block 720 todetermine one or more of the size of a training time window, size oftraining time slices within the time window, and/or type of trainingtime slices within the training time window, such as in a manner similarto that previously discussed with respect to block 553 of routine 500 ofFIGS. 5A-5B. After block 720, the routine continues to block 725 to usepartial initial state information for the target system to train themodel, including to estimate values of state variables for the targetsystem that are not known and/or directly observable, by simulatingeffects of performing control actions over the time window for each ofthe time slices, as discussed in greater detail with respect to block555 of routine 500 of FIGS. 5A-5B.

After block 725, or if it is instead determined in block 715 not totrain or re-train the model, the routine continues to block 730 toperform a piecewise linear analysis to attempt to determine a solutionfor the current model and any additional state information that wasobtained in block 710, with the solution (if determined) including oneor more proposed control action determinations for the local decisionmodule to take for a current time period, as well as in some embodimentsto use one or more model error gauges to make one or more errormeasurements with respect to the current model, as discussed in greaterdetail elsewhere. The routine then continues to block 735 to determineif the operations in block 730 determined a solution within a amount oftime allowed for the operation of block 730 (e.g., a defined subset orfraction of the current time period), and if so continues to block 740to update the current set of coupled differential equations and theresulting current model for the local decision module to reflect thesolution, with the resulting updated information provided as an outputof the routine 700.

If it is instead determined in block 735 that the operations in block730 did not determine a solution, the routine continues to block 745 todetermine if additional time is available within the current time periodfor further attempts to determine a solution, and if not continues toblock 790 to provide output of the routine 700 indicating that nosolution was determined for the current time period.

If additional time is available within the current time period, however,the routine continues to perform blocks 755-780 to perform one or morefurther attempts to identify the solution—it will be appreciated thatone or more of the operations of blocks 755-780 may be repeatedlyperformed multiple times for a given time period if sufficient time isavailable to continue further solution determination attempts. Inparticular, the routine continues to block 755 if additional time isdetermined to be available in block 745, where it determines whether themeasurements from one or more gauges indicate model error measurementsthat are over one or more thresholds indicating modifications to themodel are needed, such as based on the model error measurements from thegauges discussed with respect to block 730. If not, the routinecontinues to block 760 to determine whether there are one or more rulesor other restrictions in the current model that are available to berelaxed for the current time period (that have not previously attemptedto be relaxed during the time period, if this is not the first passthrough this portion of the routing for the current time period), and ifso continues to block 765 to relax one or more such rules or otherrestrictions and to return to block 730 to re-attempt the piecewiselinear analysis with the revised model based on those relaxed rules orother restrictions.

If it is instead determined in block 755 that the model errormeasurements from one or more of the gauges are sufficient to satisfyone or more corresponding thresholds, the routine continues instead toblock 770 to determine whether to re-train the model based on one ormore of the gauges indicating sufficient errors to do so, such as basedon accumulated errors over one or more time periods of updates to themodel. If so, the routine returns to block 720 to perform suchre-training in blocks 720 and 725, and then continues to block 730 tore-attempt the piecewise linear analysis with the resulting re-trainedmodel.

If it is instead determined in block 770 not to re-train the model (orif the model was re-trained already for the current time period and theresulting re-attempt in block 730 again failed to find a solution), theroutine continues to block 775 to determine whether the model errormeasurements from one or more of the gauges indicate a subset of one ormore rules or other restrictions in the model that potentially haveerrors that need to be repaired. If so, the routine continues to block780 to provide information to one or more users via the CDD DecisionModule Construction component, to allow the users to revise the rules orother restrictions as appropriate, although in other embodiments some orall such rule repair activities may instead be attempted or performed inan automated manner. After block 780, or if it is instead determined inblock 775 not to repair any rules, the routine continues to block 790 toprovide an indication that no solution was determined for the currenttime period. After blocks 740, 708, or 790, the routine continues toblock 799 and ends. It will be appreciated that if the routine 700 wasinstead implemented as a centralized routine that supports one or moredecision modules remote from the executing component for the routine,the routine 700 may instead return to block 703 to await furtherinformation or requests.

FIGS. 8A-8B are a flow diagram of an example embodiment of a CDDCoordinated Control Management routine 800. The routine may, forexample, be provided by execution of the component 346 of FIG. 3 and/orthe components 146 a-n of FIG. 2, such as to attempt to synchronizecurrent models and their proposed control actions between multipledecision modules that are collectively controlling a target system. Inthe illustrated embodiment of the routine, the synchronization isperformed in a pairwise manner between a particular local decisionmodule's local current model and an intermediate shared model for thatdecision module that is based on information about the current state ofone or more other decision modules, by using a Pareto game technique todetermine a Pareto equilibrium if possible that is represented in aconsensus shared model, although in other embodiments other types ofsynchronization methods may be used. In addition, in the illustratedembodiment, the routine 800 is performed in a local manner for aparticular local decision module, such as by being included within thatlocal decision module, although in other embodiments the routine 800 maybe implemented in a centralized manner to support one or more decisionmodules that are remote from a computing system implementing thecomponent for the routine and that communicate with those decisionmodules over one or more computer networks, such as with a particulardecision module indicated to be used at a particular time rather thanacting on behalf of the local decision module.

The illustrated embodiment of the routine 800 begins at block 805, whereit waits to receive information or another indication. The routinecontinues to block 810 to determine if a consensus model or otherupdated information for another decision module has been received, suchas from a copy of the routine 800 executing for that other decisionmodule, and if so continues to block 815 to use the received informationto update local intermediate shared model information for use with thelocal decision module on whose behalf the current copy of the routine800 is executing, as discussed in greater detail with respect to block830. If it is instead determined in block 810 that the information orrequest received in block 805 is not information related to one or moreother decision modules, or after block 815, the routine continues toblock 820 to determine whether to currently perform a synchronizationfor the current local model of the local decision module by usinginformation about an intermediate shared model of the local decisionmodule that includes information for one or more other decision modules,such as to do such synchronization each time that an update to the localdecision module's model is received (e.g., based on operation of theroutine 700 for a copy of the CDD Control Action Determination componentlocal to that decision module) in block 805 and/or each time thatinformation to update the local decision module's intermediate sharedmodel is received in block 805 and used in block 815, or instead asexplicitly indicated in block 805—if the synchronization is to currentlybe performed, the routine continues to block 825 and begins to performblocks 820-880 related to such synchronization activities. Otherwise,the routine continues to block 885 to perform one or more otherindicated operations as appropriate, such as to receive requests fromthe CDD system or other requestor for current information aboutoperation of the routine 800 and/or to provide corresponding informationto one or more entities (e.g., to reflect prior requests), etc.

If it is determined in block 820 that synchronization is to be currentlyperformed, such as based on updated model-related information that isreceived in block 805, the routine continues to block 825 to obtain acurrent local model for the local decision module to use in thesynchronizing, with the model including one or more proposed controlactions to perform for a current time period based on a local solutionfor the local decision module. The routine then continues to block 830to retrieve information for an intermediate shared model of the localdecision module that represents information for one or more otherdecision modules (e.g., all other decision modules) that arecollectively participating in controlling the target system, with thatintermediate shared model similarly representing one or more otherproposed control actions resulting from local solutions of those one ormore other decision modules, optionally after partial or completesynchronization has been performed for those one or more other decisionmodules between themselves.

The routine then continues to block 835 to attempt to determine aconsensus shared model that synchronizes the current model of the localdecision module and the intermediate shared model by simultaneouslyproviding solutions to both the local decision module's current modeland the intermediate shared model. In some embodiments, the operationsof block 835 are performed in a manner similar to that discussed withrespect to blocks 710-730 of routine 700 of FIG. 7A-7B, such as if thelocal model and the intermediate shared model are combined to create acombination model for whom one or more solutions are to be identified.As discussed in greater detail elsewhere, in some embodiments, the localcurrent model and intermediate shared model may each be represented by aHamiltonian function to enable a straightforward creation of such acombined model in an additive manner for the respective Hamiltonianfunctions, with the operations of routines 600 and/or 700 of FIGS. 6A-6Band 7A-7B, respectively, similarly representing the models that theyupdate and otherwise manipulate using such Hamiltonian functions.

After block 835, the routine continues to block 840 to determine whetherthe operations of block 835 succeeded in an allowed amount of time, suchas a fraction or other portion of the current time period for which thesynchronization is attempted to be performed, and if so the routinecontinues to block 845 to update both the local model and theintermediate shared model of the local decision module to reflect theconsensus shared model. As earlier noted, if sufficient time is allowedfor each decision module to repeatedly determine a consensus sharedmodel with changing intermediate shared models representing one or moreother decision modules of a collective group, the decision modules ofthe collective group may eventually converge on a single convergedshared model, although in other embodiments and situations there may notbe sufficient time for such convergence to occur, or other issues mayprevent such convergence. After block 845, the routine continues toblock 850 to optionally notify other decision modules of the consensusshared model determined for the local decision module (and/or of aconverged shared model, if the operations of 835 were a last step increating such a converged shared model), such as if each of the notifieddecision modules is implementing its own local version of the routine800 and the provided information will be used as part of an intermediateshared model of those other decision modules that includes informationfrom the current local decision module's newly constructed consensusshared model.

If it is instead determined in block 840 that a synchronization did notoccur in the allowed time, the routine continues to perform blocks860-875 to re-attempt the synchronization with one or moremodifications, sometimes repeatedly if sufficient time is available, andin a manner similar to that discussed with respect to blocks 745-780 ofroutine 700 of FIGS. 7A-7B. In the illustrated example, the routinedetermines in block 860 if additional time is available for one or moresuch re-attempts at synchronization, and if not the routine continues toblock 880 to provide an indication that no synchronization was performedwithin the allowed time. Otherwise, the routine continues to block 870to take one or more actions to perform one or more of relaxing rules orother restrictions, repairing rules or other restrictions, and/orre-training a model, with respect to one or both of the current model ofthe local decision module and/or one or more other decision moduleswhose information is represented in the intermediate shared model of thelocal decision module. If it is determined in block 870 to proceed inthis manner, the routine continues to block 875 to perform correspondingactions, sometimes one at a time in a manner similar to that discussedwith respect to routine 700, including to cause resulting updates to thecurrent model of the local decision module and/or to the localintermediate shared model of the local decision module, after which theroutine returns to block 835 to re-attempt to synchronize the localmodel and the intermediate shared model of the local decision module.

If it is instead determined in block 870 that no further actions are tobe performed with respect to relaxation, repair and/or re-training, theroutine continues instead to block 880. After blocks 850, 880 or 885,the routine continues to block 895 to determine whether to continue,such as until an explicit indication to terminate or suspend operationof the routine 800 is received, such as to reflect an end to operationof the target system and/or an end to use of the local decision moduleand/or a collective group of multiple decision modules to control thetarget system. If it is determined to continue, the routine returns toblock 805, and otherwise continues to block 899 and ends.

FIG. 9 illustrates a flow diagram of an example embodiment of a routine900 performed for a representative generic target system, with respectto interactions between the target system and one or more decisionmodules that are controlling at least a portion of the target system.The routine may, for example, be provided by execution of a targetsystem 360 and/or 370 of FIG. 3, and/or a target system 160 and/or 170of FIGS. 1 and 2, such as to implement operations specific to the targetsystem. It will be appreciated that the illustrated embodiment of theroutine focuses on interactions of the target system with the one ormore decision modules, and that many or all such target systems willperform many other operations in a manner specific to those targetsystems that are not illustrated here for the purpose of brevity.

The routine begins at block 910, where it optionally provides initialstate information for the target system to a CDD system for use in anautomated control system of the CDD system for the target system, suchas in response to a request from the CDD system or its automated controlsystem for the target system, or instead based on configuration specificto the target system (e.g., to be performed upon startup of the targetsystem). After block 910, the routine continues to block 920 to receiveone or more inputs from a collective group of one or more decisionmodules that implement the automated control system for the targetsystem, including one or more modified values for or other manipulationsof one or more control elements of a plurality of elements of the targetsystem that are performed by one or more such decision modules of theautomated control system. As discussed in greater detail elsewhere, theblocks 920, 930, 940 may be repeatedly performed for each of multipletime periods, which may vary greatly in time depending on the targetsystem (e.g., a microsecond, a millisecond, a hundredth of a second, atenth of a second, a second, 2 seconds, 5 seconds, 10 seconds, 15seconds, 30 seconds, a minute, 5 minutes, 10 minutes, 15 minutes, 30minutes, an hour, etc.).

After block 920, the routine continues to block 930 to perform one ormore actions in the target system based on the inputs received,including to optionally produce one or more resulting outputs or otherresults within the target system based on the manipulations of thecontrol elements. In block 940, the routine then optionally providesinformation about the outputs or other results within the target systemand/or provides other current state information for the target system tothe automated control system of the CDD system and/or to particulardecision modules of the automated control system. The routine thencontinues to block 995 to determine whether to continue, such as untilan explicit indication to terminate or suspend operation of the targetsystem is received. If it is determined to continue, the routine returnsto block 920 to begin a next set of control actions for a next timeperiod, and otherwise continues to block 999 and ends. As discussed ingreater detail elsewhere, state information that is provided to aparticular decision module may include requests from external systems tothe target system, which the automated control system and its decisionmodules may determine how to respond to in one or more manners.

The following sections describe a variety of specific, non-exclusiveembodiments in which some or all of the described techniques may beimplemented. It will be appreciated that particular details ofparticular embodiments may not be included in or true for allembodiments, and that the described embodiments may be implementedindividually or in combination with any and all other combinations ofother embodiments.

The following discusses several non-exclusive example embodiment, inwhich one or more specified embodiments of the Collaborative DistributedDecision system are each referred to as a Collaborative DistributedInferencing (CDI) system or a Cooperative Distributed InferencingPlatform (CDIP), in which one or more specified embodiments of theDecision Module Construction component are each referred to as the“Rules Builder” or including or being performed by a “Rule ConversionEngine” (RCE) or a “CDL Compiler” or a “Rule(s) Entry Interface” (REI)or a “Rules and Goals UI”, in which one or more specified embodiments ofthe Control Action Determination component are each referred to ashaving “chattering” subcomponents or performing “chattering”functionality or including or being performed by a “Query ResponseEngine” (QRE) or an “Optimization Element” or via“Hamiltonian-Chattering Control”, in which one or more specifiedembodiments of the Coordinated Control Management component are eachreferred to as having or performing “mean field” information orfunctionality or including or being performed by a “Tellegen” agent or a“Pareto Multi-Criteria Optimization Engine” (PMOE) or a “Paretoelement”, in which decision modules are referred to as “agents” or “peeragents” or “agent nodes” or “decision elements”, in which an automatedcontrol system may include a “cluster” of agents and in some cases isreferred to as a “distributed architecture”, in which a target system isreferred to as an “application domain”, in which a decision module'sstored information about a target system includes an “InternalHeterogeneous Database” (IHDB) and/or a “Domain Rules Knowledge Base”,in which a decision module's consensus shared model is referred to as an“internal optimum” generated using mean field information, in whichchanges propagated from one decision module to others is referred to asa “delta”, etc.

CDI is built using a Peer-to-Peer (P2P) based distributed architecturethat allows for partitioning the overall optimization problem intosmaller sub-tasks or work-loads between peer agents. CDI Peer Agents areequally privileged participants in the application domain. They areconfigured to form a peer-to-peer network of nodes. This allows eachagent in the network to solve the problem independently, using its owninternal knowledge base. Each agent also internally engages in Paretogame playing to reach internal optimum before sharing changes with otheragents. The agents then communicate the mean field changes with otheragents in the network using a gossip protocol to synchronize theirinternal mean field approximation in an eventually consistent fashion.FIG. 10 illustrates a network diagram 1000 of a portion of a distributedarchitecture of an example CDI system.

CDI Cluster Setup

When system is initially configured certain agents are tagged asseed-nodes in the network. The seed agent nodes can be started in anyorder and it is not necessary to have all the seed agent nodes running.When initializing the CDI cluster at least one seed agent node ispreferably running, otherwise the other CDI seed nodes may not becomeinitialized and other CDI agent nodes might not join the cluster.

CDI Agent Cluster Registry

Each agent has a built-in registry of active CDI agent nodes in thecluster. This agent registry is started on all CDI agent nodes and isupdated with every change in cluster membership including new agentsjoining the cluster, agents leaving the cluster, agent timeouts etc.Agents inform each other of active membership through a heartbeatmechanism. The registry is eventually consistent, i.e. changes may notimmediately be visible at other nodes, but typically they will be fullyreplicated to all other nodes after a few seconds. The changed arepropagated using the change deltas and are disseminated in a scalableway to other nodes with a gossip protocol.

CDI Cluster Agent Failure Detection

The CDI agent nodes in the cluster monitor each other by sendingheartbeats to detect if a CDI agent node is unreachable from the rest ofthe CDI cluster. The heartbeat arrival times is interpreted by animplementation of the Phi Accrual Failure Detector (Hayashibara, Défago,Yared, & Katayama, 2004).

The suspicion level of failure is given by a value called phi. The basicidea of the phi failure detector is to express the value of phi on ascale that is dynamically adjusted to reflect current networkconditions.

The value of phi is calculated as:

Phi=−log 10(1−F(timeSinceLastHeartbeat))

F is the cumulative distribution function of a normal distribution withmean and standard deviation estimated from historical heartbeatinter-arrival times.

CDI Cluster Agent Network Partition

Once a CDI agent node becomes network partitioned it will stop receivingmean field updates from the other agents in the Cluster. This however,should not prevent it from continuing to perform local optimizationsbased on its internal knowledge base, local mean field and sensor inputsthat it will continue to receive.

The Rules Builder contains the following components:

-   -   Rules Entry Interface    -   Validator    -   CDL Compiler    -   Step Processor    -   Workflow Remote Control    -   Compiled Rules Engine Artifact

And interacts directly with the following components:

-   -   Master Agent.

The Master Agent is responsible for interaction with all Chatteringcomponents, including the following:

-   -   System Trainer & Bootstrapper    -   Runner

The System also utilizes a Persistent Store to access state andnecessary models/data. FIG. 11 illustrates a network diagram 1100 of aportion of an example Rules Builder component.

Rules Entry Interface:

The Rules Entry Interface is responsible for receiving rules in a syntaxfamiliar to a domain expert. The entry interface is a text entry toolwhere a domain expert would insert the rules, goal and systeminformation (rules script(s)) to provide continuous control suggestionsfor a given problem. The entered rules script(s) would be composed inthe CDI Domain-Specific Language (DSL). The CDI DSL facilitatesfunctional translation of the entered rules script(s) into a problemdefinition which houses the control definitions (measurable variablewith ranges allowed and any associated constraints on the controlvariable).

For example:

In the entry interface using the CDI DSL one would specify an upper andlower bound using the following syntax first defining a rule, with theparameters and the rule logic that evaluates against the system state.

-   -   rule id: “max production”, params: [“production”], {it <=15.0}    -   rule id: “min production”, params: [“production”], {it >=−50.0}    -   control var: “production”, min: “min production”, max: “max        production”

In the entry interface using the CDI DSL one would specify the dynamics(variables that change with relation to other measurable variables andthe definition of this change)

-   -   delta var: “inventory”, params: [“production”, “demand”], {p,        d->p−d}

In the entry interface using the CDI DSL one would specify the goal (anobjective i.e., to minimize or maximize a relationship between thecontrol and controllable dynamics).

-   -   goal objective: “minimize”, params: [“inventory”, “production”],        {inventory,        production->((inventory−10)*(inventory−10))+(production*production)}

Finally, the entry interface is also responsible for facilitating adomain expert to provide certain settings that are used by the system aswell as providing user workflow steps. The settings provided representkey: value terms used by the system and are available throughout by theSettings Provider encapsulated within the Rules Engine. An domain expertmight write initial values of the following form:

-   -   settings <<[initialState: “20”, initialCoState: “0”,        terminalCoState: “0”, numChatteringLevels: “9”, horizon: “3”,        delta: “0.1”, iterations: “15”]

CDL Compiler:

The CDL Compiler is responsible for the conversion of said rules intoevaluatable constraints, system information and an optimization goal.The CDI Compiler translates the entered script(s) into a problemdefinition. A problem definition is composed of labeled rules. A rule isa tuple comprised of a unique name and a Term. A Term is an evaluatablefunction where the logic was authored as previously described above, theinput to which contains a representation of the system state (StateMap).

Validator:

The Validator is responsible for the validation of converted constraintswith regards to restrictions the optimization problem needs to satisfyas a prerequisite to its solution and/or reaching a solution qualitythreshold.

The converted constraints defined in the problem definition, along withthe settings provider, is validated for controllability, observability,stability and goal completeness.

-   -   controllability—this check ensures that every control variable        has a defined rule relating it to one or more dynamic state        variables. The above example satisfies this check since our        single control variable ‘production’ is used in the dynamic        definition of ‘inventory’;    -   observability—checks the statemap against the defined rules in        the script(s). An observability check does not impede the        solution of a problem, however if failing, represents that our        problem may not be well defined;    -   stability—this check ensures that our system will converge over        time on a solution, which can be tested in the testing stage;        and    -   goal completeness—ensures that every control variable appears in        the goal.

Compiled Rules Engine:

The conversion of rule scripts into mathematical expressions that can beyielded through an explicit contract to the chattering agent for use insolving the optimization goal, manifests in the compiled artifact thatwe label the Compiled Rules Engine

The Rules Engine defines and implements several interfaces for acquiringthe values of these mathematical expressions, as later described withrespect to “chattering” components. The Rules Engine is compiled as anartifact available to the Chattering component by the Compiler and madeavailable for use in continuous processing. FIG. 12 illustrates anetwork diagram 1200 of example sub-components of an example RulesBuilder component, in which compilation is dependent upon successfulValidation of the observability, completeness, controllability, andworkflow step validation. Any errors/warnings are returned to the domainexpert for correction. Once corrections/improvements are complete, a newCompiled Rules Engine artifact is produced along with new commandsdelivered to the Step Processor.

Steps Processor and Remote Control:

Furthermore, the rules builder is also responsible for facilitating theworkflow steps an individual domain expert would need to undertakebefore deploying such a system. These steps include:

-   -   training an optimization system using the converted rules, with        bootstrapped sensor data;    -   testing the resulting control actuator inputs (the end results        of the chattering optimization); and    -   persisting the trained model for use in the running of the        overall system.

The workflow component allows for training and testing a system. This isintegrated into the DSL in the form of a workflow step builder.

For example,

workflow {    train.onComplete {   persist: model, ‘/tmp/model’      run.withSensorData(‘/tmp/simulatedData’).after(10 minutes)    {         persist: results, ‘/tmp/results’          shutdown       }    }}

Here a domain expert instructs the default bootstrapper to be used inloading predefined sensor data, train the system on this data, persistthe trained model output to the file ‘/tmp/model’, then once trained tobegin running the system (as a single agent). The inputs are tested byanalyzing the results from the run. The artifacts and model informationare persisted to a persistent store. The Steps Processor facilitates thedispatch of interpreted workflow steps to the Remote Control fordelegation to the Master Agent of a running system for processing.

FIG. 13 illustrates a network diagram 1300 of interactions of portionsof a Rules Builder component with an executing agent, including theSteps processor, the Remote Control and interactions with a runningagent (the Step Processor handles interpretation of workflow steps, andthe steps are delivered for dispatch by the Remote Control, which passescommands to the Master Agent; the Master Agent handles communicationbetween a running system's controlled elements and the Remote Control,and examples of the Controlled Elements are the Trainer and Runner).

FIG. 14 illustrates a network diagram 1400 of interactions of portionsof a Rules Builder component with chattering components, including aRunning Agent that delivers messages and interacts with the chatteringcomponents to facilitate training and running the system (the Trainerpersists its model once training has completed; The Trainer invokes aBootstrapper for acquisition of training data; The Trainer utilizes theChattering Library; The Trainer utilizes the Chattering Library;Chattering utilizes the compiled rules; Chattering persists its controlsuggestions; Bootstrapper persists training data).

With respect to the discussion below, FIG. 15 illustrates a diagram 1500of a sliding window for use with chattering, and is referred to in thetext for the example embodiment as “FIG. 1”, while FIG. 16 illustrates adiagram 1600 of using Lebesgue integration to approximate a controltrajectory for chattering, and is referred to in the text for theexample embodiment as “FIG. 2”.

The following discusses an example implementation that may be used forsuch chattering.

The following discusses further details regarding possible use with suchchattering.

In the chattering algorithm, there are two timing parameters, the lengthof the window T, and the time slice Δ (where NΔ=T). For accuracypurposes, these two parameters should be sufficiently small, but forcomputational efficiency, these two parameters should be large.Therefore, we want to keep the parameters as large as possible, as longas the error is tolerable.

The incremental state equation (for δz) relies on the Jacobian matrixevaluated at the beginning of the time window. To characterize theerror, we compare two linear systems, one that has a constant matrix,denoted A(t₀), and the other has a time varying matrix, denoted A(t).

Consider both linear systems:

{dot over (x)}=A(t)x(t)  (1)

{dot over (y)}=A(t ₀)y(t)  (2)

with the same initial conditions, x(t₀)=y(t₀).

The solution to (1) is of the form

x(t)=φ(t,t ₀)x(t ₀)

where φ(t, t₀) satisfies several properties,

{dot over (φ)}(t,t ₀)=A(t)φ(t,t ₀)

φ(t ₀ ,t ₀)=1

φ⁻¹(t,t ₀)=φ(t ₀ ,t).

The solution to (2) is of the form

y(t)=e ^(A)(t−t ₀)x(t ₀).

The error due to approximating A(t) with A(t₀) is:

E(t)=φ(t,t ₀)−e ^(A(t−t) ⁰ ⁾.

Taking the derivative yields

{dot over (E)}(t)={dot over (φ)}(t,t ₀)−A(t ₀)e ^(A(t−t) ⁰ ⁾

and replacing {dot over (φ)}(t, t₀) with A(t)φ(t, t₀) yields

{dot over (E)}(t)=A(t)φ(t,t ₀)−A(t ₀)e ^(A(t−t) ⁰ ⁾

and substituting for φ(t, t₀) yields

$\begin{matrix}{{\overset{.}{E}(t)} = {{{A(t)}\left( {{E(t)} + ^{A{({t - t_{0}})}}} \right)} - {{A\left( t_{0} \right)}^{A{({t - t_{0}})}}}}} \\{= {{{A(t)}{E(t)}} + {\left( {{A(t)} - {A\left( t_{0} \right)}} \right){^{A{({t - t_{0}})}}.}}}}\end{matrix}$

This provides the error as a function of (A(t)−A(t₀)), and if theeigenvalues of A(t) for all t have real values less than zero, the erroris linearly proportional to (A(t)—A(t₀)).

The procedure to choose T is to ask the customer what size of error istolerable.

The customer may say an error of 20% is tolerable, in which case thewindow length T can increase until reaching the associated threshold.The measure of the error is related to the difference,

$\sum\limits_{i = 1}^{I}{\frac{\partial F}{\partial z}\alpha_{i}{_{t}{- {\sum\limits_{i = 1}^{I}{\frac{\partial F}{\partial z}\alpha_{i}}}}}_{t_{0}}}$

The size of the time slice Δ is related to the magnitude of the largesteigenvalue of

$\frac{\partial F}{\partial z}.$

Let |λ|=√{right arrow over (λ_(real) ²+λ_(imag) ²)} and suppose |λ|_(i)has the largest value. Then

$\Delta = {\frac{1}{{\lambda }_{i}}.}$

The following discusses further details regarding an example ofsynchronizing a decision module's model and current information withthat of information from one or more other decision modules.

In the multi-agent self-organizing architecture of CDI, all agentssynchronize in some way. This is analogous in the worst case to themany-body problem, which Isaac Newton first formulated, and isunsolvable for three or more bodies. However, good approximations forsystems of more than two bodies are computationally tractable. Theapproach in CDI is to solve a series of two-body problems that emulate amean field aggregation, and update sequentially through a Pareto game.

Consider N agents, and each agent i, i=1, . . . , N, has its ownoptimization problem with criterion, state and control:

Min_(u) _(i) J _(i)(x _(i) ,u _(i))

s·t·{dot over (x)} _(i) =f(x _(i) ,u _(i))

The N problem in a single optimization problem is not algorithmicallysolvable, but it is possible to solve algorithmically aPareto-optimization problem with two players (the Pareto game), andapproximate the solution to the N player problem. The two-agent (agent 1and agent 2) Pareto-optimization problem is

Min_(u) ₁ _(,u) ₂ _(,α) ₁ _(,α) ₂ α₁ J ₁(x ₁ ,u ₁)+α₂ J ₂(x ₂ ,u ₂)

subject to

{dot over (x)} ₁ =f(x ₁ ,u ₁)

x ₂ =f(x ₂ ,u ₂)

α₁(t)+α₂(t)=1

The CDI approach is that each agent i plays a two-agent game with theCDI Mean Field agent composed of the criteria from all other agentsexcept agent i. Instead of solving the two-agent optimization problemdirectly, we convert the formulation to a Hamiltonian problem (usingPontryagin's minimum principle).

The Hamiltonians are additive for Pareto optimization. Therefore, we canadd local Hamiltonians for individual agents to create an aggregate meanfield Hamiltonian.

Let H_(i) be the local Hamiltonian for agent i and let H_(i) ^(MF) bethe mean field

Hamiltonian composed of the Hamiltonians for all other agents, excludingi. That is, the mean field Hamiltonian for agent i is a functional formof the Hamiltonians of the other agents.

Then the two-agent optimization problem is

Min_(u_(i), u_(i)^(MF), α₁, α₂)α₁H_(i)(x_(i), u_(i)) + α₂H_(i)^(MF)(x_(i)^(MF), u_(i)^(MF))

subject to the state and costate equations of the combined Hamiltonian,and

α₁(t)+α₂(t)=1.

The state consists of [x_(i),x_(i) ^(MF)]^(T), the costate is[p_(i),p_(i) ^(MF)]^(T), and the control is [u_(i),u_(i) ^(MF)]^(T). Theinitial conditions for x_(i) ^(MF) and p_(i) ^(MF) come from theprevious solution, and u_(i) is the local solution from the previouspass.

The solution to the two-agent Pareto game provides α₁(t) and α₂(t), andthe total Hamiltonian for agent i is updated:

H _(i) ^(T)=α₁ H _(i)+α₂ H _(i) ^(MF)

The modified Hamiltonian for agent i is constructed by projecting thetotal Hamiltonian into the local state space. The modified mean fieldHamiltonian is constructed by projecting the total Hamiltonian into themean field state space.

Now, the another agent plays the game, solving the two-agent Paretogame, and updating locally, and each agent updates its mean field agentwhen it is ready to play its game.

The following discusses further details regarding example embodiments.

Overview

The Cooperative Distributed Inference (CDI) platform is a uniqueadvanced technology offering to enable near-optimal and near-real-timedecision making on vast amount of heterogeneous and distributedinformation, in complex knowledge-based decision support systems,combining absolute, hard and soft rules to handle various requirementsfrom natural or governing laws, policies, and best practices. FIG. 17illustrates a diagram 1700 of various interactions of different portionsof a CDI system.

The CDI platform features a Distributed Architecture (DA) for resolvingqueries by accessing information from both an Internal HeterogeneousDatabase (IHDB) and external data sources referred to as Sensors. CDIutilizes a network of computing devices as its nodes—called DecisionElements (DE)—that cooperate to resolve queries given to them.

The Decision Elements (DE) in a given DA can work together to reach bestoutcome for the whole group, i.e., reaching Pareto Efficiency (alsoreferred to as Pareto equilibrium)—a stable state where no change by anyindividual can be made to make the sum of whole group better.

Each DE can solve the problem independently if provided with completeknowledge and data. DE solves the query with a very unique approach,using Optimal Control Theory, starting with a technique called analyticcontinuation (transforming query and rules into differential equationswhose dependent variables represent internal variables and parameters ofthe rules), then using its own internal knowledge to solve the query,providing an outcome.

In distributed environments, group of Decision Elements synchronize inan iterative process utilizing updates from other DEs and provide afinal result, via a Pareto multi criteria optimization strategy.

The CDI platform needs rules, not necessarily exact criteria, to reachthe objective state, producing the near-optimal solution. This is a veryattractive feature when exact quantitative criteria cannot be providedin advance due to uncertainty or other reasons.

Platform Features

CDI is perfect for big data analytics, supporting many data types:

-   -   Structured: databases, ontologies    -   Semi-structured: spreadsheets, CSV/TSV, forms, emails    -   Unstructured: web pages, blogs, text documents    -   Symbolic: business rules, math formulas

CDI's distributed computing architecture also make it very scalable tohandle large amount (peta-size) of heterogeneous data ingestion whileperforming real-time analytic results even in microseconds (depending inpart on resources available and the complexity of queries).

Rules Support

CDI can integrate different types of business rules: absolute, hard andsoft rules, from natural or government laws, operational or policyrequirements, and practice guidelines. Absolute rules and hard rulesalways take logic value 0 (false) or 1 (true) when instantiated. Softrules, however, may take any value in the interval [0, 1], or moregenerally more than two values. Absolute rules reflects a must-satisfysituation, such as FDA/USDA requirements; hard rules are operationalrequirements such as “no serious persistent side effects”, but which maybe temporarily relaxed in specific situations; and soft rules can comefrom guidelines or experiences, such as “better not give drug XYZ todiabetes patients with heart problems”. FIG. 18 illustrates a diagram1800 of examples of different types of rules. Please note that allchaining of the rules may happen automatically in a dynamic mannerduring the optimization process. CDI handles this complexity with saidtechniques above by solving a distributed continuous-space optimizationproblem.

Self-Adapting and Learning

CDI platform features a self-learning design. As CDI converts theoriginal query solving into an optimal control problem, it can usefeedbacks from the environment (external sensor or internal updates frompeer decision elements) to refine its internal model: aHamilton-Jacobi-Bellman equation will be updated to reflect newinformation and automatically form soft-rule like constraintsinternally. The process to update internal mathematical model is similarto reconstructing 3D images from CT or MRI imaging by tomographicreconstruction, but used for dynamic systems.

Scalability and Performance

CDI platform enables functionality to:

-   -   Integrate data that may include 1,000,000+ variables and        100,000+ of constraints    -   Provide data integration over evolving distributed network    -   Specify queries over a broad range of languages    -   Specify queries of a broad range of complexity    -   Provide best known response to queries at the local level    -   Operate in a variety of environments including cloud-based or        local deployments    -   Real-time processing of queries

Comparisons

CDI stands out in being able to do the following things:

-   -   Approximate the optimal solution of NP-hard problems (such as        planning and scheduling) by mapping criteria and constraints        onto a continuous space and solve it as an Optimal Control        problem with polynomial-time algorithms. The solutions offered        by CDI may be near optimal rather than exactly optimal but the        running time will be greatly shortened and some large-scale        intractable problems can become solvable by CDI.    -   Blend adaptive learning together with rules composition.        Traditional rule engines might support hard and soft rules        (constraints), but the scoring mechanism and weights need be        manually adjusted to meet the goals, while in CDI, via feedback        mechanism, these weights can be optimized as internal        configurations.

The unique abilities make CDI an ideal choice for intelligent decisionsupport systems.

Sample Applications

CDI platform works great in areas where there are a lot of heterogeneousdata, government compliances and business requirements, such ashealthcare and energy.

Clinical Auto-Coding Application

A Clinical Auto-Coding (C.A.C.) application can be used to detectmedical under-coding, over-coding, and miscoding, highlighting potentialopportunities where higher billing is justified. It automaticallygenerates clinical codes, including ICD-10 directly from clinicalencounter notes such as physician notes, lab results, and dischargerecords, while supporting workflows accommodating the roles played byadministrators, coders and doctors in coding. FIG. 19 illustrates anexample user interface 1900 related to medical/clinical auto-coding.C.A.C. incorporates user feedback to learn coding best practices as itprocesses records. With its sophisticated adaptive technology, C.A.C.improves over time, optimizing coding for each organization to improvethe efficiency, accuracy and revenue capture of the medical codingactivity

Clinical Intelligence Web Services

CDI powers the following intelligent healthcare services that can beeasily integrated:

-   -   Clinical Record Intelligence—For analysis of EMRs (electronic        medical records) and encounter notes, identification of        actionable clinical terms and concepts in those documents. The        services can extract concepts, understand issues, and analyze        documents.    -   Clinical Coding Intelligence—For inferring clinical codes from a        set of clinical concepts, grouping codes, correlating codes,        analyze documents, audit and justification, and compliance.    -   Patient-Centric Intelligence—Secure delivery of end-user        applications to predict, recommend, and explain personalized        actions to improve patient outcomes. It provides a patient        centric view, can provide medical risk prediction and proactive        monitoring for patients, automated data abstraction, actionable        recommendations, and more.

All these web services run in secure cloud with full compliancemeasurement in place.

Fraud, Waste and Abuse Detection

In healthcare, CDI helps assess and monitor risk of medical fraud, wasteand abuse (FWA) by uncovering providers, and to a lesser extentpharmacies, who are suspected of having committed fraud via a variety ofschemes. Similar application domains include financial frauds.

By ingesting a wide variety of data that is difficult to link (includinglive public data), a FWA service analyzes data to find evidence andpatterns of fraud, and provides a dashboard application foragents/detectives to prioritize and act on the discovered suspectedcases of medical fraud. The service features strict and fuzzyrules-based detection as well as automatic pattern discovery, and runsin real-time and continuous mode to support proactive monitoring andaction.

Energy Intelligence

Smart Grid can involve uncertain bi-directional exchange in distributedgrids, so intelligent control can be used to provide activesynchronization between the network of element controllers and theoutside grid management system allows high quality of service to bemaintained in a cost-effective manner, hence the dynamics can be learnedfrom sensory observations. CDI enables distributed micro-grid control,supports inductive modeling with “soft” rules that are learned andcontinuously updated, to optimize the grid.

Distributed Architecture (DA)

The DA is a network of interacting components called decision elements(DEs). The DEs collaborate in the resolution of a query posed by one ofthem. The DA's block diagram is shown in FIG. 18, where Sensors refer toexternal input data in general. There is an additional translation layerto process external data to be consumed by DEs.

Decision Element (DE)

A decision element is a higher-level functional component solvingqueries locally. It can have subcomponents such as a programmable searchengine, internal heterogeneous database, Inference engine, Inferencerule base, API/user interface, and network interface. A decision elementis capable of providing a quick and near-optimal solution to a complexquery with complete input of data.

Internal Heterogeneous Database (IHDB) and External Knowledge Base (EKB)

IHDB is data preprocessed and stored by a specific decision element(DE). FIG. 20 illustrates a diagram 2000 of some components of a CDIsystem, including Knowledge Bases containing data sources ranging fromdomain knowledge to general facts. IHDB encodes knowledge into knowledgecomponents (KC's). Each KC is used and updated by a DE in the DA, andmultiple KCs may share rules.

External Knowledge Base (EKB) refers to data, including rules, as inputinto specific decision elements, such as patient's body temperature,blood pressure, or instantiated rule to determine if patient'scholesterol level is high. EKB can also contain communicated informationfrom other DEs. Domains for variables include: real, complex, integer,binary numbers and symbolic token on finite domains.

Interfacing Components

The following components act between users and the data via API and/orGUI.

-   -   Rule Entry Interface—It provides the entry of rules into the        IHDB, validates the specification of rules before insertion, and        route the rules to the appropriate DE for insertion to their        respective knowledge components.    -   Sensor Ingestion Interface—A sensor is a machine or service        where external data exist. Sensor Ingestion Interface (SII)        enables the system to add or remove a sensor, poll a sensor and        submit data to the network.    -   Query Language Interface—Query Language Interface accepts the        query, submits it to the system (Distributed Architecture) and        provides response. It is an API and flexible UI can be built on        top of it.

Minimization Function Generator (MFG)

The minimization function generator converts a query to a minimizationfunction (i.e. analytic continuation). This is useful because theproblem is converted from search problem in a large discrete space (likegraph search problems, which are usually NP-complete) into anoptimization problem in continuous space (polynomial algorithms exist).An analogy is NP-hard integer programming, while continuous linearprogramming has a very efficient solution.

Query Response Engine (QRE)

The Query Response Engine is the core of the whole system responsiblesolving the query (locally). A mathematical model is constructed basedon these equations obtained from previous steps, containing the currentstate and object state (goal state). The continuous-space optimizationautomatically handles forward and backward rule chaining by moving alongthe trajectory toward target state. It also manages uncertainty bykeeping a large set of possible states and reducing the solution spaceonly when more information becomes available. Standard optimizationtechniques (e.g. Newton-Raphson Method) can be employed to solve theproblem.

Feedbacks and updates (data from EKB or other DE in the architecture)will be used to refine the mathematical model over time; therefore, thecore engine is self-adapting.

Pareto Multi-Criteria Optimization Engine

The Pareto Multi-Criteria Optimization Engine (PMOE) is the aggregationstep where all DE in the network settle to obtain a good stablesolution—a state where no improvements can be made to any individual DEwithout reducing the whole team's performance—a state belongs ParetoOptimal Set in Game Theory. It is like each DE is playing a game andthey communicate and interact and work together to make the best outcomefor the criteria (query). To efficiently synchronize all decisionelements, Mean Field Theory is applied for dimension reduction usingknowledge obtained. FIG. 21 illustrates a diagram 2100 of performingPareto processing for use with mean field techniques, including to reachPareto efficiency.

Data Exchange Specifications

The system can take many different data types via ingestion API and hasadapters from different public data sources. The supported data typesinclude common ones from XML, CSV, TSV, SQL, Spreadsheet, JSON, andmore. OData support is also available. Output types can be API (XML orJSON), as well as exporting to CSV, SQL or directly to services such asSOLR and Cassandra.

Running Environment

CDI can run in the cloud as Software as a Service platform. We canprovide whole end-to-end support by setting up the infrastructure in aVirtual Private Cloud or deploy and configure it with the clusterclients provide. The system features SaaS architecture and provides APIsto be used by third parties. For advanced integration requests,Java/Scala and Python libraries are available.

To sum up, CDI platform utilizes many advanced techniques from MeanField Theory, Lagrangian and Hamiltonian functions, Pareto Optimal Set,Gauge Theory, and so on derived from modern mathematics, quantumphysics, optimal control and game theory to achieve high performance. Inthe detailed computation process, lots of transformation, approximation,optimization, caching and other advanced computing techniques are usedto improve accuracy, speed and scalability. Due to this unique approach,CDI is able to solve complex decision making problem in a smoothly,efficient manner and achieve near optimal results.

The Cooperative Distributed Inference (CDI) platform is a uniqueadvanced technology offering to enable near-optimal and near real-timedecision making on vast amount of heterogeneous and distributedinformation, in complex knowledge-based decision support systems,combining absolute, hard and soft rules to handle various requirementsfrom natural or governing laws, policies, and best practices.

Each agent in the Distributed Architecture is a Decision Element thatcan solve the problem independently, provided with knowledge base anddata. In a very unique manner, the agent uses Optimal Control Theory toobtain a near optimal solution, starting with a technique calledanalytic continuation (transforming query and rules into differentialequations whose dependent variables represent internal variables andparameters of the rules), then using its own internal knowledge to solvethe problem as an optimization problem in continuous space, to allow forefficient solving.

The outcome will be fed back to the Rules Editor and optimizationprocess (the Chattering algorithm) to enable automatic adjustment ofweights of soft rules (constraints) and achieve optimal score of theobjective function.

A CDI agent is an independent decision element that can take in sensordata (input from the environment, streaming or in batches), as well asthe knowledge-bases (rules composed by domain experts, includingabsolute, hard and soft rules) to generate a set of partial differentialfunctions (Lagrangian constraints), through continualization. Similarly,the objective is also converted into a minimization function, as part ofthe Lagrangians which will be solved via the Hamilton-Jacobi-Bellmanequation. Each agent will talk to other agents via a “mean field”abstraction layer to greatly reduce the communication and computationoverhead, and incorporate additional information to reach the globaloptimal state (a stable, near optimal set of states across all agents).Finally, the agent exercises control over the system. FIG. 22illustrates a network diagram 2200 of an example decision module agent.

Various domain knowledge is captured from experts in the form of DomainSpecific Language. Some are constraints; logic forms need be convertedto Boolean equations; and variables in soft rules take values [0 . . .1]. This creates a set of equations to be solved by the optimizationalgorithm. The optimization process takes data from a time range andfinds the best state configuration to reach optimality. It starts with a“learning” process to find a good initial configuration by taking in ashort history of data, before processing real-time streams. FIG. 23illustrates a network diagram 2300 of an example of offline workflowsfor knowledge capture.

Each CDI agent computes a “mean field” view of the system via itsneighbors, and responds to queries with the latest updates. Theapproximate mean field view of a group greatly reduces computationaldimensions. The agents synchronize with others and understand the globalstate via the mean-field approximation. They engage in games to reachPareto Optimal (also referred to as Pareto equilibrium)—the best output.FIG. 24 illustrates a network diagram 2400 of an example of workflowsfor mean field computation and Pareto Optimal.

An example home solar micro-grid system illustrates one exampleembodiment of a CDI application or automated control system, which takesthe sensor data from the solar panel (in the house), substation, andpower network, and decides whether or not to fulfill the utilityrequests in real time using a set of complex rules. FIG. 25 illustratesa network diagram 2500 of an example of an automated control system fora home solar micro-grid electrical generating system.

FIGS. 26-28 provide further details regarding operations of portions ofexample CDI systems and their sub-components, and FIGS. 29A-29Killustrate examples of using a CDI system to iteratively determinenear-optimal solutions over time for controlling a target system indiagrams 2900A-2900K. In particular, FIG. 26 illustrates a furtherdiagram 2600 of workflow and components of a portion of a CDI system,FIG. 28 illustrates a diagram 2800 of an overview workflow for a portionof a CDI system, and FIG. 27 illustrates a diagram 2700 of workflow foran inference process portion of a CDI system.

Further details related to an example CDI system are shown below andincluded in provisional U.S. Patent Application No. 62/015,018, filedJun. 20, 2014 and entitled “Methods And Systems For CooperativeDistributed Inferencing,” which is hereby incorporated by reference inits entirety. In addition, further details related to example details ofusing gauges to perform model error measurements are included inprovisional U.S. Patent Application No. 62/182,796, filed Jun. 22, 2015and entitled “Gauge Systems,” which is also hereby incorporated byreference in its entirety. Furthermore, further details related toexamples of using the CDD system in particular manners with particulartypes of target systems are included in provisional U.S. PatentApplication No. 62/182,968, filed Jun. 22, 2015 and entitled“Applications Of Cooperative Distributed Control Of Target Systems,”which is also hereby incorporated by reference in its entirety—theexample types of target systems discussed include, for example, avehicle being controlled, controlling inventory management, controllinga micro-grid electrical generation facility, controlling activities in aday trader financial setting, etc.

Current notation Comments t Notation for algorithmic time. q(t) Notationof canonical coordinate vector for entire system. q Notation ofcanonical coordinate vector dropping time dimension. q^((ƒ)) Notation ofcanonical coordinate vector for specific function ƒ. {dot over (q)}Notation of first time derivative of canonical coordinate vector.{umlaut over (q)} Notation of second time derivative of canonicalcoordinate vector. h Notation of HEAD of Horn clause. φ(q) Notation ofgeneric proposition. σ(q) Notation of generic proposition alternate toφ. T_(i) Notation of the TV of a soft rule. {hacek over (r)}(q; φ, σ)Generic equational form relating two propositions. {hacek over (φ)}(q)Notation of the equational form of φ(q). φ_(Q)(q) Notation forproposition defined by the query. {hacek over (φ)}_(Q)(q) Notation forequation defined by the query. J(q) Notation for minimization functionfor the query. L Notation for static Lagrangian L_(k) ^((o, T)) Notationfor total static Lagrangian for DE_(k). q {p_(α)} u^((k)) H_(k) ^((o))Primary Hamiltonian for the absolute rules for DE_(k). H_(k) ^((A))Hamiltonian for the Tellegen agent of the total Hamiltonian's rules.H_(k) ^((T)) Total Hamiltonian for DE_(k).

Overview

This document introduces and specifies the architecture for theCooperative Distributed Inferencing Platform (CDIP). The primaryinstance of this is the Distributed Architecture (DA) for resolvingqueries by accessing both an Internal Heterogeneous Database (IHDB)populated by a special class of Horn Clause rules and external datasources referred to as sensors.

The architecture implements a network of active devices at its nodes.These devices are called Decision Elements (DE's). The DE's cooperate inthe resolution of a query posed to one or several of them. The DE's in agiven DA are referred to as the team.

Every DE in a team is programmed to transform rules in its domain,determined by a posed query, into an ordinary differential equation(ODE), whose dependent variables represent internal variables andparameters. The dependent variables include unknowns of the query posedto the DE. The DE's in the architecture are synchronized via a Paretomulti criteria optimization strategy.

The components of the CDIP include:

-   -   Application requirements that the system is designed to        accommodate.    -   Functional requirements that satisfy the application        requirements and pertain directly to the construction and        operation of the system components.    -   Subcomponents which are necessary to implement the functional        requirements    -   Limitations which highlight noteworthy constraints which are        inherent the specified implementation of the architecture.    -   Architectural flow describes key aspects of the architecture        that indicate how the system is to be constructed given the        specified essence and key behavior of the subcomponents.    -   Software realization of the architecture describes the key        pieces of software necessary for system implementation.    -   Data describes the kinds of data the system is expected to        accept as input and produce as output.    -   Data exchange protocols reference key data types and structures        that need to be exchanged across the system and the protocols        for exchange.    -   Environment describes the particulars of the environments that        the system will be able to operate in and therefore should be        tested in.    -   Testing describes how the system should be tested given the data        and operating environments.

Subcomponents

Subcomponents are fundamental parts of the architecture that performparticular roles. This section contains descriptions for each of thesubcomponents of the architecture. The subcomponents are:

-   -   The Distributed Architecture (DA).    -   The Internal Heterogeneous Database (IHDB).    -   The Rule Entry Interface (REI).    -   The Rule Editor (RE).    -   The External Knowledge Base (EKB).    -   The Sensor Ingestion Interface (SII).    -   The Rule Conversion Engine (RCE).    -   The Decision Element (DE).    -   The Query Language Interface (QLI).    -   The Minimization Function Generator (MFG).    -   The Query Response Engine (QRE).    -   The Pareto Multi-Criteria Optimization Engine (PMOE).

Distributed Architecture (DA)

The DA is a platform of interacting components called DE's. The DE'scollaborate in the resolution of a query posed by one of them. The DE'simplement a distributed, dynamic optimization process, herein referredas the optimization process (OP). OP implements an optimization processthat computes an answer to the active queries as a function of datastored in two categories of repositories: IHDB and EKB's. Theserepositories of the data needed to implement OP given a query.

The EKB's are a collection of public or private repositories ofknowledge relevant to the DE posing a query. A DE has a list of externalrepositories (LER). Each entry in an LER includes 1) a protocol, 2) aheading sub-list, and 3) a translation grammar. Each protocol entryprescribes the access procedure to the corresponding knowledgerepository. Each heading sub-list entry contains a summary of theknowledge contents of the corresponding repository. Finally, eachtranslation grammar entry provides a procedure for converting knowledgeelements of the corresponding repository in to the rule representationin the IHDB of the DE.

Functional characteristics of this architecture and in particular, theDE's, IHDB, and the sensors are described, including the followingconcepts:

-   -   The DA    -   A process for resolving queries by accessing the IHDB and        External Knowledge Bases (EKB's) through sensors    -   The constitution of DE's    -   A query and corresponding rules transformation into an ODE    -   The orchestration of a team of DE's through a Pareto multi        criteria optimization strategy

The Internal Heterogeneous Database (IHDB)

Composition of IHDB as a Set of Knowledge Components

The IHDB encodes knowledge about the implemented application. The IHDBis divided into knowledge components (KC's). Each KC is consulted andupdated by a DE in the DA. Any pair of KC's may have an overlapping setof rules by which they operate, but there is no a priori constraint onintersections or inclusion. The collection of KC's constitutes theexisting knowledge of the system.

Algorithmic Formulation of a Rule

A KC is a collection of rules, written in a restrictive Horn clauseformat. The rules are logic entities. When a rule is instantiated, ithas a logic value. The logic values a rule can have are taken from theinterval [0, 1]. The entire system of rules is evaluated using variablesand parameters which are collectively referred to as the generalizedcoordinates of the system and are indexed as follows:

q(t)={q ⁽¹⁾(t), . . . ,q ^((N))(t)}.  (3.2-1)

The time argument t refers to the algorithmic time of the system whichmeans that it has no other meaning than as a continuous index withrespect to the evolution of the system. There is therefore norequirement that it correspond to a physical aspect of the systemalthough this may naturally occur. Physical time may be representedspecifically by a canonical coordinate of choice q^((i))(t).Alternatively, we may refer to the q's without expressly stating theindependent time argument and write

q(t)={q ⁽¹⁾ , . . . ,q ^((N))}.  (3.2-2)

Then we should also note that the time derivatives are referred tonotationally as

$\begin{matrix}{{\overset{.}{q} = \frac{{q(t)}}{t}},\text{}{\overset{¨}{q} = \frac{^{2}{q(t)}}{t^{2}}}} & \left( {3.2\text{-}3} \right)\end{matrix}$

These coordinates are referred to variously as q depending on thecontext and the expected arguments of the function to which they areapplied. When it is necessary to distinguish between more than one q inequational form we generally write q_(f) of where f denotes thereference function or appropriate domain. Typically, we assume withoutloss of generality the entire set of canonical coordinates q is anargument to any function, term or proposition. In practice, we mayfurther assume it is possible to apply the particular requiredcoordinates as need to mathematical construct in question.

The rules in each knowledge component are of three types: absoluterules, hard rules, and soft rules. Absolute rules and hard rules takelogic value 0 (false) or 1 (true) when instantiated. Soft rules take anyvalue in the interval [0, 1].

The format of the restrictive Horn Clauses in the IHDB is illustrated inEquation 3.2-2. A Horn Clause is an object composed of two objects aHEAD and a BODY connected by backward Implication (

). The logic implication transfers the logic value of the BODY to theHEAD. If the rule is an absolute rule or a hard rule, the logic value is1 (if the BODY is logically true) or 0 (if the BODY is logically false).If the rule is a soft rule, the logic value transferred by the body isany number in [0, 1].

The HEAD is a data structure composed of two objects: A name, h, and alist of arguments described by the argument vector q=(q⁽¹⁾, q^((n))).The list of arguments includes variables and parameters. The variablestake values in the domain of the rule and the parameters are constantspassed to the rule and unchanged by the instantiation of the rule. Thedomain of the rule is a set of values that each of its variables cantake. In general, variables can take values over numerical or symbolicdomains. As an example, a symbolic domain can be a list of diseases. Anumeric domain can be a set of pairs of numbers representing bloodpressure.

For the applications of CU, the domains for variables are: real numbers,complex numbers (floating point and floating point complex numbers),integer numbers, binary numbers and symbolic token on finite domains.

The BODY of a clause is a data structure composed of one or more terms,denoted φ_(i)(q). The composition operation is extended- and, denotedby: ̂. The extended- and works as a regular and in absolute rules andhard rules and as a functional product² on soft rules.

A rule with a head but not a body is called a fact. A fact's truth valueis determined on the basis of the instantiation of its variables.

Each term in the body of a rule is an extended disjunction (or denotedby v) of sub-terms. The v operator behaves like the standard- or forabsolute and hard rules and behaves in a functional form, describedlater, when connecting sub-terms encoding heads of soft rules.

A sub-term is either the HEAD of a rule, a relation or a truth valuation(TV). When it is a HEAD it may have the same name as the one in the HEADof the rule but with different arguments. This provides a recursivemechanism for rule evaluation.

When a rule has a sub-term that is the head of another rule it is saidthat the two rules are chained together by the corresponding sub-term.Note that a rule can be chained to several rules via correspondingsub-terms.

Constraint Domains

Constraint domains augment the BODY clause of Horn clauses to facilitatedynamic programming. Constraints are specified as a relationship betweenterms. Define the relationship between two terms

φ(q)relσ(q).  (3.2-4)

as a member of the following set

rel ε{=,≠,≦,≧,statistical propagation}.  (3.2-5)

A relation can be of two types numeric or symbolic. Numeric relationsestablish equational forms between two functional forms. (For theinitial phase only polynomial and affine linear functional forms will beconsidered.)

In general, an equational form is a set of one or more relations. Fornumeric relations, φ(q) rel σ(q), rel ε{=, ≠, ≦, ≧, <, >, statisticalpropagation}. The table below gives the relations considered and theirsymbols.

TABLE EE Numeric Relation Symbol Code Form Equality = φ = σ Disequation≠ φ \= σ Less-inequality < φ < σ Less-Equal ≦ φ =< σ Great-equality >φ > σ Great- inequality ≧ φ >= σ

The adopted code forms are the ones used in constraint logicprogramming.

A symbolic relation can be of two types: inclusion and constraint.Inclusion relations are of the form:

qεSet  (3.2-6)

Where x is a variable or a parameter, ε is the inclusion symbol and Setis a set of symbolic forms or a set of numbers or a composite set of theform shown in the table below.

TABLE FF Composite Set Symbol Code Form Intersection ∩ Set1/\Set2 Union∪ Set1\/Set2 Complement \ \Set

Constraint forms of the symbolic relational type may be one or a set ofthe forms presented in the table below For numeric relations, φ(q) relσ(q), rel ε {=, ≠, ⊂, ⊃, ⊂, ⊃}.

TABLE GG Symbolic Relation Symbol Code Form Equal = φ# = σ Not Equal ≠φ# \= σ Is Contained ⊂ φ# < σ Contains ⊃ φ# > σ Is Contained or Equal ⊂φ# =< σ Contains or Equal ⊃ φ# >= σ

A TV is either a variable or a constant with values in the interval [0,1]. The TV of a Horn Clause that is an absolute rule or a hard rule canonly take two values: 1 or 0. The TV when instantiated is 0 or 1. If theTV for an absolute or hard rule is 1, the rule is said to be in inactivestate; if the TV is 0, the rule is said to be in active state.

The TV, T_(i), of a soft rule satisfies

0≦T _(i)≦1.  (3.2-7)

If T_(i) above satisfies,

T _(i) ≧T _(threshold)  (3.2-8)

the soft clause is said to be in inactive state. If

T _(i) <T _(threshold),  (3.2-9)

the soft clause is said to be in active state, where T_(threshold) is aconstant in [0, 1] defined for each soft clause. The default value is0.5.

This concludes the description of the knowledge representation. Theinstantiation process of the goal in a DE, as function of its knowledgebase, is carried out by the inference engine of the DE. This process isthe central component of CU and will be described later on the document.

Summary of Terminology

The following table summarizes the terminology we have just reviewed.

TABLE HH Reference term Definition proposition Defined as a construct asin the prepositional calculus where the proposition takes on the valueof true or false. term Recursively according to its assigned sub-term.sub-term A sub-term may be a Horn clause, a relation between two othersub-terms or an extended truth valuation depending on the context ofabsolute, hard or soft rules. In the case of absolute and hard rules itmay be evaluated as a proposition. In the case of soft rules it takes avalue on the interval [0, 1] and is considered to be active or true inthe case that it exceeds its specific threshold. Horn clause Adisjunction of terms with at most one positive term. definite clause AHorn clause with exactly one positive term. goal clause A Horn clausewith no positive terms. fact A definite clause with no negative terms.head The positive term of a definite clause. inactive state The casewhen a rule will not apply for constrained optimization. active stateThe case when a rule will apply for constrained optimization. truthvalue, TV The value that is used to determine whether a rule is activeor inactive.

Horn Clause Example

The following example illustrates a Horn clause:

has_fever(name, temperature, white_count, heartrate, blood_pressure)

(temperature>37)

((heartrate≧70)

bp(name, temperature, blood_pressure)

wc(name, white_count))  (3.2-10)

The clause establishes under which conditions the patient of name name,has a fever. The variables in clause are:

-   -   name, temperature, white_count, heartrate, blood_pressure.

When instantiated they represent, respectively, the name of the patient,his current body temperature, his white blood cell count, his heart raterange, and his blood pressure.

The clause body includes other clauses: bp (blood pressure) and wc(white count).

This completes the specification of the rule-based framework. The nextstep is to specify a complete process for converting all rules of thisform to a set of equations.

Rule Entry Interface (REI)

The Rule Entry Interface provides a mechanism for:

-   -   Providing an API for the entry of rules into the IHDB.    -   Validating the specification of rules to be inserted into the        IHDB.    -   Routing the rules to the appropriate DE's for insertion to their        respective KC's.

Rule Editor (RE)

The Rule Editor allows users to specify rules associated with thesystems to be interrogated.

External Knowledge Base (EKB)

-   -   It may be distributed or not    -   It may be persisted or not    -   It may be persisted locally or remotely to an agent    -   It may or may not be architecturally co-located with the IHDB    -   A sensor may include any source of data used by the agent    -   It may use various types of buses for data communication    -   Sensors may or may not be co-located with agents/DEs

Rule Conversion Engine (RCE)

The rule conversion engine converts rules of the IHDB into equations.

Method for Specification of a Simple Term as an Equation

Consider the term φ(q) with the following truth assignment.

$\begin{matrix}{{\phi (q)} = \left\{ \begin{matrix}T & {q \in _{\phi}} \\F & {q \notin _{\phi} \Subset }\end{matrix} \right.} & \left( {3.6\text{-}1} \right)\end{matrix}$

Then we can define the set of arguments which yield positive truthassignment.

{qε

_(φ)|φ(q)←T}.  (3.6-2)

Define the corresponding equation {hacek over (φ)}(q) of the term φ(q)as

$\begin{matrix}{{\overset{\bigvee}{\phi}(q)} = \left\{ \begin{matrix}1 & \left. {\phi (q)}\leftarrow T \right. \\0 & \left. {\phi (q)}\leftarrow F \right.\end{matrix} \right.} & \left( {3.6\text{-}3} \right)\end{matrix}$

Then extend the range of {hacek over (φ)}(a) to the closed unit interval

{hacek over (φ)}(q)→[0,1].  (3.6-4)

Revisiting the taxonomy of absolute, hard and soft rules, we recognizethat hard and soft rules (terms in this example) can take values alongthe interval

0≦{hacek over (φ)}(q)≦1.  (3.6-5)

And for absolute rules we add the constraint {hacek over (φ)}(q)→{0,1}

{hacek over (φ)}(q)(1−{hacek over (φ)}(q))=0.  (3.6-6)

Conversion of Fundamental Clauses of Propositional Calculus to Equations

Define the following notation for the propositional calculus.

TABLE II Symbol Function

And

Or

Implication ~ Not ∃ Exists ∀ All Fuzzy rule

Theorem 3.6.1.

Given the method for the specification of equations from propositions,we prove the following transformations.

TABLE JJ Proposition Equation ~φ(q) 1 − {hacek over (φ)}(q) φ(q)

 σ(q) {hacek over (φ)}(q) · {hacek over (σ)}(q) φ (q)

 σ(q) {hacek over (φ)}(q) + {hacek over (σ)}(q) − {hacek over (φ)}(q) ·{hacek over (σ)}(q) φ(q)

 σ(q) 1 − {hacek over (φ)}(q) + {hacek over (φ)}(q) · {hacek over(σ)}(q) φ₁(q)

 φ₂(q)

 . . .

φ_(k−1)(q)

 φ(q)

 φ(q)${\overset{ˇ}{\overset{\sim}{\varphi}}\left( {n,q} \right)} = \frac{\overset{ˇ}{h}\left( {{n - 1},q} \right)}{{{\overset{ˇ}{\overset{\sim}{\sigma}}\left( {n,q} \right)}{\overset{ˇ}{\overset{\sim}{\varphi}}\left( {{n - 1},q} \right)}} - 1}$(tail recursive)

Proof by enumeration for equational representation of conjunction

Define the function {hacek over (r)}(q; φ, σ) which represents theequation corresponding to conjunction (

). Verify by enumeration the correspondence of the mathematical equationvalues corresponding to the mapping T→1 and F→0.

TABLE KK φ(q)

σ(q) {hacek over (r)}(q; φ, σ) = {hacek over (φ)}(q) · {hacek over(σ)}(q) T T T 1 = 1 · 1 T F F 0 = 1 · 0 F F T 0 = 0 · 1 F F F 0 = 0 · 0

Proof by enumeration for equational representation of disjunction

Define the function {hacek over (r)}(q; φ, σ) which represents theequation corresponding to disjunction (

). Verify by enumeration the correspondence of the mathematical equationvalues corresponding to the mapping T→1 and F→0.

TABLE LL φ(q)

σ(q) {hacek over (r)}(q; φ, σ) = {hacek over (φ)}(q) + {hacek over(σ)}(q) − {hacek over (φ)}(q) · {hacek over (σ)}(q) T T T 1 = 1 + 1 − 1· 1 T T F 1 = 1 + 0 − 1 · 0 F T T 1 = 0 + 1 − 0 · 1 F F F 0 = 0 + 0 − 0· 0

Proof by enumeration for equational representation of negation

Define the function {hacek over (r)}(q; φ, σ) which represents theequation corresponding to negation (

). Verify by enumeration the correspondence of the mathematical equationvalues corresponding to the mapping T→1 and F→0.

TABLE MM φ(q) ~φ(q) {hacek over (r)}(q; φ, σ) = 1 − {hacek over (φ)}(q)T F 0 = 1 − 1 F T 1 = 1 − 0

Proof by enumeration for equational representation of implication

Define the function {hacek over (r)}(q; φ, σ) which represents theequation corresponding to disjunction (

). First note the equivalence of

φ(q)

σ(q) and ˜φ(q)

σ(q).  (3.6-7)

Verify by enumeration the correspondence of the mathematical equationvalues corresponding to the mapping T→1 and F→0.

TABLE NN ~φ(q)

σ(q) {hacek over (r)}(q; φ, σ) = 1 − {hacek over (φ)}(q) + {hacek over(φ)}(q) · {hacek over (σ)}(q) T T T 1 = 1 − 1 + 1 · 1 T F F 0 = 1 − 1 +1 · 0 F T T 1 = 1 − 0 + 0 · 1 F T F 1 = 1 − 0 + 0 · 0

Proof for equational representation of tail recursion

Tail recursion is propositionally defined as

φ(q)

φ₁(q)

. . .

φ_(k−1)(q)

φ(q)  (3.6-8)

where s represent the current state. To develop an equationalrepresentation of the recursive formulation, first define the generalfunction {tilde over (φ)}(n, q) where n represents the n^(th) iterationof the tail recursion ad {tilde over (φ)}(n, q) is the logicalconsequent. Then rewrite the above formulation using the recursive step.

{tilde over (φ)}(n,q)

{tilde over (φ)}₂(n,q)

. . .

{tilde over (φ)}_(k−1)(n,q)

{tilde over (φ)}(n−1,q)

{tilde over (φ)}(n,q)  (3.6-9)

Define

{tilde over (σ)}(n,q){tilde over (=)}{tilde over (φ)}₁(n,q)

{tilde over (φ)}₂(n,q)

. . .

{tilde over (φ)}_(k−1)(n,q){tilde over (r)}(n−1,q){tilde over (=)}{tildeover (σ)}(n,q)

{tilde over (φ)}(n−1,q)  (3.6-10)

-   -   Then the tail recursion is rewritable as

{tilde over (σ)}(n,q)

{tilde over (φ)}(n−1,q)

{tilde over (φ)}(n,q){tilde over (r)}(n−1,q)

{tilde over (φ)}(n,q).  (3.6-11)

According to the equational representation of implication, let

{hacek over (h)}(n−1,q)=1−{hacek over ({tilde over (σ)}(n,q)·{hacek over({tilde over (φ)}(n−1,q)+{hacek over ({tilde over (σ)}(n,q)·{hacek over({tilde over (φ)}(n−1,q)·{hacek over ({tilde over (φ)}(n,q).  (3.6-12)

Since by definition {circumflex over ({tilde over (r)}(n−1,q)={circumflex over ({tilde over (φ)}(n, q)·{circumflex over ({tildeover (φ)}(n−1, q). Then

$\begin{matrix}{{\overset{\bigvee}{\overset{\sim}{\phi}}\left( {n,q} \right)} = \frac{{\hat{h}\left( {{n - 1},q} \right)} + {{\hat{\overset{\sim}{\sigma}}\left( {n,q} \right)} \cdot {\hat{\overset{\sim}{\phi}}\left( {{n - 1},q} \right)}} - 1}{{\hat{\overset{\sim}{\sigma}}\left( {n,q} \right)} \cdot {\hat{\overset{\sim}{\phi}}\left( {{n - 1},q} \right)}}} & \left( {3.6\text{-}13} \right)\end{matrix}$

with boundary condition n=0.

Converting Rules Based System of Inference to the Problem of ConstrainedMinimization

Consider the following example.

-   -   Converting rules to constraints

The preceding discussion has established an algorithm for convert rulesof the form

h(q)

φ₁(q)

φ₂(q)

. . .

φ_(m)(q)  (3.6-14)

To constraints of the form

{hacek over (h)}(q)={hacek over (φ)}₁(q)·{hacek over (φ)}₂(q)· . . .·{hacek over (φ)}_(m)(q).  (3.6-15)

Decision Element (DE)

A diagram of the Decision Element Architecture is shown in illustration◯◯. It is composed six elements:

-   -   Programmable search engine (PSE)    -   Internal heterogeneous database (IHDB)    -   Inference engine (IE)    -   Inference rule base (IRB)    -   API/user interface    -   Network interface (NI)

List of External Repositories (LER)

A DE has a List of External Repositories (LER). Each entry in an LERincludes 1) a protocol, 2) a heading sub-list, and 3) a translationgrammar. Each protocol entry prescribes the access procedure to thecorresponding external knowledge repository. Each heading sub-list entrycontains a summary of the knowledge contents of the correspondingrepository. Finally, each translation grammar entry provides a procedurefor converting knowledge elements of the corresponding repository in tothe rule representation in the IHDB of the DE.

Programmable Search Engine

The programmable search engine implements a standard hashing algorithmfor detecting active rules as a function of the current instantiation ofthe variables in a variable buffer (VB) of the IE, and the contents ofthe active rule buffer (ARB). The VB contains the variables that formpart of the query and all additional variables incorporated to thisbuffer during the inference process (IP). The VB includes all relevantdata from the EKB beneficial to perform the query. The IP will bedescribed below. The ARB contains all the currently active rules in theIP.

The search hashing algorithm is characterized by the search rules in theInference Rule Base.

Internal Heterogeneous Database

The IHDB is the repository of the application clauses associated withthe DE. These encode the domain of knowledge characterizing theexpertise of the DE. For example in a medical application, a decisionelement may deal with expertise on heart illnesses, and thecorresponding clauses might encode diagnoses and treatments for thesediseases.

Inference Engine

The IE encodes an algorithm, the IP, for assigning values to thevariables appearing in the query.

Inference Rule Types

The DE incorporates inference rules (IR) that are a collection of rulesfor transforming and inferring instantiations of the goal. These rulesprovide the Inference Engine with directives for processing databaserules to give a satisfactory instantiation to a given query or torequest additional information so that a satisfactory instantiation canbe generated. They are organized according to their functionality asfollows. (See Illustration PP)

Equation Rules

These rules include the formal rules for inference. This includes allrules for natural language modeling from first principles.

Optimizer Rules

These rules include rules for finding the interior point inoptimization.

Search Rules

These rules include rules for identifying the nature of insufficientpotential. The goal is to apply these rules to acquire additionalinformation required to satisfy the optimization goal.

Adaptation Rules

Adaptation rules are used to update the soft rules to relax them furtherto reduce the complexity and constrains of the optimization problem. Theadaptation also serves to update the search rules to improve informationacquisition.

Language Statistics and Pattern Rules

These rules embody the machine learning models.

Network Rules

These rules define how information is distributed over the network andwhat information is available from which resources.

Hybridization Rules

The rules define how other rules may be combined.

User Interface

The UI provides the utilities for entering queries, pragma rules,displaying query answers, status and for general interaction with theIE.

Network Interface

The NI provides a generic mechanism for interacting with other DE's viaa procedure termed companionship. The companionship procedure implementsthe active coupling for the cooperation of the DE's in query resolution.This procedure is not hierarchical and implements a Pareto Agreement setstrategy as the mechanism for Ca

Query Language Interface (QLI)

Information about the QLI is available elsewhere herein.

Process for Determining Active Constraints

The process for determining active constraints is available elsewhereherein.

Minimization Function Generator (MFG) and Determining Active Constraints

The minimization function generator converts a query to a minimizationfunction.

Again, we assume without loss of generality the entire set of canonicalcoordinates q is an argument to any proposition φ_(i). In practice, wemay further assume it is possible to apply the particular requiredcoordinates as need to the proposition or function in question. Then letφ^(k) be the set of propositions associated with DE_(k) in the contextof query Q. These propositions are composed of the propositionassociated with the query φ_(Q)(q), and other propositions φ_(i)(q),comprising the constraints of the system. The proposition φ_(Q)(q)associated with a given query Q can be converted to an equation {hacekover (φ)}_(Q)(q). Queries that are satisfiable specify a set.

{q|φ _(Q)(q)←T}  (3.10-1)

Similarly, a satisfied query represented as an equation is also a set

{q|{hacek over (φ)} _(Q)(q)=1}.  (3.10-2)

Relaxing the values that {hacek over (φ)}_(Q)(•) can take to include theunit interval so that soft rules are incorporated yields the followingconstrained optimization expression. Let J(q)=({hacek over(φ)}_(Q)(q)−1)². Then specify the optimization

$\begin{matrix}{\min\limits_{q}{J(q)}} & \left( {3.10\text{-}3} \right)\end{matrix}$

Subject to:

-   -   1. {hacek over (φ)}_(Q)(q)≦1    -   2. {hacek over (φ)}_(Q)(q)≧0    -   3. A knowledge base on the set {{hacek over (φ)}₁(q), . . . ,        {hacek over (φ)}_(n)(q), . . . , {hacek over        (φ)}_(n+s)(q)}⊂{hacek over (φ)}^((k)) which represents a further        set of active constraints specific to the problem:        -   a. {hacek over (φ)}_(i)(q)≧0 for 1≦i≦n,        -   b. {hacek over (φ)}_(i)(q)≦1 or, equivalently −({hacek over            (φ)}_(i)(q)−1)≧0 for 1≦i≦n,        -   c. and in the case of absolute rules {hacek over            (φ)}_(l)(q)(1−{hacek over (φ)}_(l)(q))=0 for n<l≦n+s.

Introduce the indicator functions

$\begin{matrix}{V_{{\overset{\bigvee}{\phi}}_{i}}^{-} = \left\{ {\begin{matrix}0 & {{{\overset{\bigvee}{\phi}}_{i}(q)} \geq 0} \\\infty & {{{\overset{\bigvee}{\phi}}_{i}(q)} < 0}\end{matrix}{and}} \right.} & \left( {3.10\text{-}4} \right) \\{V_{{\overset{\bigvee}{\phi}}_{i}}^{+} = \left\{ \begin{matrix}0 & {{1 - {{\overset{\bigvee}{\phi}}_{i}(q)}} \geq 0} \\\infty & {{1 - {{\overset{\bigvee}{\phi}}_{i}(q)}} < 0}\end{matrix} \right.} & \left( {3.10\text{-}5} \right)\end{matrix}$

which yields the two logarithmic barrier functions

{hacek over (V)} _({hacek over (φ)}) _(i) ⁻=−log({hacek over(φ)}_(i)(q))  (3.10-6)

and

{hacek over (V)} _({hacek over (φ)}) _(i) ⁺=−log(1−{hacek over(φ)}_(i)(q)).  (3.10-7)

According to the method of Lagrange multipliers, combine this with theequality constraints to form the static Lagrangian function

$\begin{matrix}{{{\mathcal{L}\left( {{q;{\overset{\bigvee}{\phi}}_{Q}},{\overset{\bigvee}{\phi}}^{(k)},\omega_{1}^{( + )},\ldots \mspace{14mu},\omega_{n}^{( + )},\omega_{n + 1}^{( - )},\ldots \mspace{14mu},\omega_{2\; n}^{( - )},\omega_{{2\; n} + 1}^{(\lambda)},\ldots \mspace{14mu},\omega_{{2\; n} + s}^{(\lambda)},\omega_{{2\; n} + s + 1}^{(Q)},\omega_{{2\; n} + s + 2}^{(Q)}} \right)} = {{{\overset{\bigvee}{\phi}}_{Q}(q)} + {\sum\limits_{i = 1}^{n}\; \left\lbrack {{\omega_{i}^{( + )}{\overset{\bigvee}{V}}_{{\overset{\bigvee}{\phi}}_{i}}^{+}} + {\omega_{n + i}^{( - )}{\overset{\bigvee}{V}}_{{\overset{\bigvee}{\phi}}_{i}}^{-}}} \right\rbrack} + {\sum\limits_{l = 1}^{s}\; {\omega_{{2\; n} + 1}^{(\lambda)}{{\overset{\bigvee}{\phi}}_{l}(q)}\left( {1 - {{\overset{\bigvee}{\phi}}_{l}(q)}} \right)}} - {\omega_{{2\; n} + s + 1}^{(Q)}{\log \left( {{\overset{\bigvee}{\phi}}_{Q}(q)} \right)}} - {\omega_{{2\; n} + s + 2}^{(Q)}{\log \left( {1 - {{\overset{\bigvee}{\phi}}_{Q}(q)}} \right)}}}},} & \left( {3.10\text{-}8} \right)\end{matrix}$

the roots of which can be found using a formulation of Newton-Raphson.Since

here includes absolute, hard and soft rules we may call it the totalstatic Lagrangian for DE_(k) and refer to it as

_(k) ^((T)).

Construct Equations of Motion

Information for equations of motion is available elsewhere herein.

Query Response Engine (QRE) which Includes Process for ConstructingDifferential Equations

Application of Newton-Raphson

Consider a continuous analog of the independent variables of

(•)

$\begin{matrix}{q = {{q(t)} = \begin{bmatrix}{q^{(1)}(t)} \\\vdots \\{q^{(v)}(t)}\end{bmatrix}}} & \left( {3.12\text{-}1} \right)\end{matrix}$

where each of the v total independent variables of

(•) is mapped to its corresponding position in q(t), the column vectorthat is represented with a lower-case q. To reiterate, the independentvariable t refers algorithmic time as opposed to physical time which mayalso be represented in the system. The corresponding unconstrainedoptimization goal can be written as

$\begin{matrix}{\min\limits_{q_{1},\ldots \mspace{14mu},q_{v}}{\mathcal{L}\left( {{q^{(1)}(t)},\ldots \mspace{14mu},{q^{(v)}(t)}} \right)}} & \left( {3.12\text{-}2} \right)\end{matrix}$

so that ∇L(q)

$\begin{matrix}{{{\nabla{\mathcal{L}\left( {q(t)} \right)}} = {\begin{bmatrix}\frac{\partial\mathcal{L}}{\partial q^{(1)}} \\\vdots \\\frac{\partial\mathcal{L}}{\partial q^{(v)}}\end{bmatrix} = {\begin{bmatrix}{\nabla\mathcal{L}_{1}} \\\vdots \\{\nabla\mathcal{L}_{v}}\end{bmatrix} = 0}}},} & \left( {3.12\text{-}3} \right)\end{matrix}$

with positive definite Hessian matrix

$\begin{matrix}{{\nabla^{2}{\mathcal{L}\left( {q(t)} \right)}} = {\begin{bmatrix}\frac{\partial\mathcal{L}}{{\partial q^{(1)}}{\partial q^{(1)}}} & \cdots & \frac{\partial\mathcal{L}}{{\partial q^{(1)}}{\partial q^{(v)}}} \\\vdots & \ddots & \vdots \\\frac{\partial\mathcal{L}}{{\partial q^{(v)}}{\partial q^{(1)}}} & \ldots & \frac{\partial\mathcal{L}}{{\partial q^{(v)}}{\partial q^{(v)}}}\end{bmatrix} = {\quad{\begin{bmatrix}{\nabla\mathcal{L}_{11}} & \ldots & {\nabla\mathcal{L}_{1\; v}} \\\vdots & \ddots & \vdots \\{\nabla\mathcal{L}_{v\; 1}} & \ldots & {\nabla\mathcal{L}_{vv}}\end{bmatrix} = {> 0.}}}}} & \left( {3.12\text{-}4} \right)\end{matrix}$

Write the recursion for Newton's method

q _((k+1))(t)=q _((k))(t)−(∇²

(q _((k))(t)))⁻¹∇

(q _((k))(t)).  (3.12-5)

This is equivalently rewritten

$\begin{matrix}{\frac{{q_{({k + 1})}(t)} - {q_{(k)}(t)}}{\delta} = {{- \frac{1}{\delta}}\left( {\nabla^{2}{\mathcal{L}\left( {q_{(k)}(t)} \right)}} \right)^{- 1}{{\nabla{\mathcal{L}\left( {q_{(k)}(t)} \right)}}.}}} & \left( {3.12\text{-}6} \right)\end{matrix}$

Via continualization we approximate the derivative

$\begin{matrix}{{\overset{.}{q}(t)} = {\frac{{q(t)}}{t} = {{- \left( {\nabla^{2}{\mathcal{L}\left( {q(t)} \right)}} \right)^{- 1}}{{\nabla{\mathcal{L}\left( {q_{(k)}(t)} \right)}}.}}}} & \left( {3.12\text{-}7} \right)\end{matrix}$

Translation of Inverted Matrix

Consider M, an invertible and positive definite matrix. Then we make thefollowing provable assertions.

-   -   1. A^(T)A is symmetric.    -   2. -A^(T)A has negative eigenvalues.

Define

$\begin{matrix}{\frac{{M(t)}}{t} = {{{- A^{T}}{{AM}(t)}} + A^{T}}} & \left( {3.12\text{-}8} \right)\end{matrix}$

Then as t→∞, M(t)→A⁻¹=∇²

(q_((k))(t))⁻¹. Using (3.12-3) and (3.12-4) approximate {dot over(q)}(t) by rewriting the derivative in the context of M(t). This yieldsthe following two equations.

$\begin{matrix}{{\overset{.}{q}(t)} = {{{- {M(t)}}{\nabla{L\left( {q(t)} \right)}}} = {{\begin{bmatrix}m_{11} & \ldots & m_{1\; v} \\\vdots & \ddots & \vdots \\m_{v\; 1} & \ldots & m_{vv}\end{bmatrix}\begin{bmatrix}{\nabla\mathcal{L}_{1}} \\\vdots \\{\nabla\mathcal{L}_{v}}\end{bmatrix}} = \begin{bmatrix}{{m_{11}{\nabla\mathcal{L}_{1}}} + \ldots + {m_{1\; v}{\nabla\mathcal{L}_{v}}}} \\\vdots \\{{m_{v\; 1}{\nabla\mathcal{L}_{1}}} + \ldots + {m_{vv}{\nabla\mathcal{L}_{v}}}}\end{bmatrix}}}} & \left( {3.12\text{-}9} \right) \\\begin{matrix}{\frac{{M(t)}}{t} = {{{- \left( {\nabla^{2}{\mathcal{L}\left( {q(t)} \right)}} \right)^{T}}\left( {\nabla^{2}{\mathcal{L}\left( {q(t)} \right)}} \right){M(t)}} + \left( {\nabla^{2}{\mathcal{L}\left( {q(t)} \right)}} \right)^{T}}} \\{= {- {{{\begin{bmatrix}{\nabla\mathcal{L}_{11}} & \ldots & {\nabla\mathcal{L}_{v\; 1}} \\\vdots & \ddots & \vdots \\{\nabla\mathcal{L}_{1\; v}} & \ldots & {\nabla\mathcal{L}_{vv}}\end{bmatrix}\begin{bmatrix}{\nabla\mathcal{L}_{11}} & \ldots & {\nabla\mathcal{L}_{1\; v}} \\\vdots & \ddots & \vdots \\{\nabla\mathcal{L}_{v\; 1}} & \ldots & {\nabla\mathcal{L}_{vv}}\end{bmatrix}}\begin{bmatrix}m_{11} & \ldots & m_{1\; v} \\\vdots & \ddots & \vdots \\m_{v\; 1} & \ldots & m_{vv}\end{bmatrix}}\begin{bmatrix}{\nabla\mathcal{L}_{11}} & \ldots & {\nabla\mathcal{L}_{v\; 1}} \\\vdots & \ddots & \vdots \\{\nabla\mathcal{L}_{1v}} & \ldots & {\nabla\mathcal{L}_{vv}}\end{bmatrix}}}} \\{= {- \begin{bmatrix}{{\nabla\mathcal{L}_{11}^{2}} + \ldots + {\nabla\mathcal{L}_{v\; 1}^{2}}} & \ldots & {{{\nabla\mathcal{L}_{11}}{\nabla\mathcal{L}_{1\; v}}} + \ldots + {{\nabla\mathcal{L}_{v\; 1}}{\nabla\mathcal{L}_{vv}}}} \\\vdots & \ddots & \vdots \\{{{\nabla\mathcal{L}_{11}}{\nabla\mathcal{L}_{1\; v}}} + \ldots + {{\nabla\mathcal{L}_{v\; 1}}{\nabla\mathcal{L}_{vv}}}} & \ldots & {{\nabla\mathcal{L}_{1\; v}^{2}} + \ldots + {\nabla\mathcal{L}_{vv}^{2}}}\end{bmatrix}}} \\{{\begin{bmatrix}m_{11} & \ldots & m_{1\; v} \\\vdots & \ddots & \vdots \\m_{v\; 1} & \ldots & m_{vv}\end{bmatrix} + \begin{bmatrix}{\nabla\mathcal{L}_{11}} & \ldots & {\nabla\mathcal{L}_{v\; 1}} \\\vdots & \ddots & \vdots \\{\nabla\mathcal{L}_{1\; v}} & \ldots & {\nabla\mathcal{L}_{vv}}\end{bmatrix}}} \\{= {- \begin{bmatrix}\begin{matrix}{{\left( {{\nabla\mathcal{L}_{11}^{2}} + \ldots + {\nabla\mathcal{L}_{v\; 1}^{2}}} \right)m_{11}} + \ldots +} \\{{\left( {{{\nabla\mathcal{L}_{11}}{\nabla\mathcal{L}_{1\; v}}} + \ldots + {{\nabla\mathcal{L}_{v\; 1}}{\nabla\mathcal{L}_{vv}}}} \right)m_{v\; 1}} +} \\{\nabla\mathcal{L}_{11}}\end{matrix} & \ldots & \begin{matrix}{{\left( {{\nabla\mathcal{L}_{11}^{2}} + \ldots + {\nabla\mathcal{L}_{v\; 1}^{2}}} \right)m_{1\; v}} + \ldots +} \\{{\left( {{{\nabla\mathcal{L}_{11}}{\nabla\mathcal{L}_{1\; v}}} + \ldots + {{\nabla\mathcal{L}_{v\; 1}}{\nabla\mathcal{L}_{vv}}}} \right)m_{vv}} +} \\{\nabla\mathcal{L}_{v\; 1}}\end{matrix} \\\vdots & \ddots & \vdots \\\begin{matrix}{{\left( {{{\nabla\mathcal{L}_{11}}{\nabla\mathcal{L}_{1\; v}}} + \ldots + {{\nabla\mathcal{L}_{v\; 1}}{\nabla\mathcal{L}_{vv}}}} \right)m_{11}} + \ldots +} \\{{\left( {{\nabla\mathcal{L}_{1\; v}^{2}} + \ldots + {\nabla\mathcal{L}_{vv}^{2}}} \right)m_{v\; 1}} +} \\{\nabla\mathcal{L}_{1\; v}}\end{matrix} & \ldots & \begin{matrix}{{\left( {{{\nabla\mathcal{L}_{11}}{\nabla\mathcal{L}_{1\; v}}} + \ldots + {{\nabla\mathcal{L}_{v\; 1}}{\nabla\mathcal{L}_{vv}}}} \right)m_{1\; v}} + \ldots +} \\{{\left( {{\nabla\mathcal{L}_{1\; v}^{2}} + \ldots + {\nabla\mathcal{L}_{vv}^{2}}} \right)m_{vv}} +} \\{\nabla\mathcal{L}_{vv}}\end{matrix}\end{bmatrix}}} \\{= \begin{bmatrix}{\nabla m_{11}} & \ldots & {\nabla m_{1\; v}} \\\vdots & \ddots & \vdots \\{\nabla m_{v\; 1}} & \ldots & {\nabla m_{vv}}\end{bmatrix}}\end{matrix} & \left( {3.12\text{-}10} \right)\end{matrix}$

The approximation proceeds as follows, applying the Magnus expansion tocompute the integral:

-   -   1. Fix M(0)=∇²        (q(t)) and =∇²        (q(t)).    -   2. Use the variation of constants formula to solve

M(T)=e ^(−[∇) ²

^((q(T))]) ² ^(t) M(0)+[∫₀ ^(T) e ^(−[∇) ²

^((q(τ))]) ² ^((T−τ)) dτ]∇ ²

(q(T))

The following is computation flow, flowing down unless otherwiseindicated.

Process for Determining Dynamic Lagrangian Via Hemholtz Equations

Given

$\begin{matrix}{{{G_{i}\left( {\overset{¨}{q},\overset{.}{q},q} \right)} = {{{\sum\limits_{j = 1}^{n}\; {{W_{i,j}\left( {\overset{.}{q},q} \right)}\left( {\overset{.}{q},q} \right){\overset{¨}{q}}^{(j)}}} + {K_{i}\left( {\overset{.}{q},q} \right)}} = 0}}{{j = 1},\ldots \mspace{14mu},n}} & \left( {3.12\text{-}11} \right)\end{matrix}$

If the three conditions

$\begin{matrix}{{\frac{\partial G_{i}}{\partial{\overset{¨}{q}}^{(j)}} = \frac{\partial G_{j}}{\partial{\overset{¨}{q}}^{(i)}}},{{\frac{\partial G_{i}}{\partial{\overset{.}{q}}^{(j)}} + \frac{\partial G_{j}}{\partial{\overset{.}{q}}^{(i)}}} = {\frac{}{t}\left( {\frac{\partial G_{i}}{\partial{\overset{¨}{q}}^{(j)}} + \frac{\partial G_{j}}{\partial{\overset{¨}{q}}^{(i)}}} \right)}},{{\frac{\partial G_{i}}{\partial q^{(j)}} - \frac{\partial G_{j}}{\partial q^{(i)}}} = {\frac{1}{2}\frac{}{t}\left( {\frac{\partial G_{i}}{\partial{\overset{.}{q}}^{(j)}} - \frac{\partial G_{j}}{\partial{\overset{.}{q}}^{(i)}}} \right)}},} & \left( {3.12\text{-}12} \right)\end{matrix}$

with i, j=1, . . . , n hold, then

$\begin{matrix}{{{{\sum\limits_{j = 1}^{n}{\frac{\partial^{2}L}{{\partial{\overset{.}{q}}^{()}}{\partial{\overset{.}{q}}^{(j)}}}{\overset{¨}{q}}^{(j)}}} + \frac{\partial^{2}L}{{\partial q^{(j)}}{\partial{\overset{.}{q}}^{()}}} - \frac{\partial L}{\partial q^{()}}} = G_{i}},{i = 1},\ldots \mspace{14mu},n} & \left( {3.12\text{-}13} \right)\end{matrix}$

This is a second order, linear hyperbolic differential equation on theLagrangian L. It can be solved efficiently by the method ofcharacteristics.

Let

$\begin{matrix}\begin{matrix}{{G\left( {\overset{¨}{q},\overset{.}{q},q} \right)} = \begin{bmatrix}{q(t)} \\{\overset{.}{q}(t)} \\{\overset{.}{M}(t)}\end{bmatrix}} \\{= \begin{bmatrix}q^{(1)} \\\vdots \\q^{(v)} \\{{m_{11}{\nabla\mathcal{L}_{1}}} + \ldots + {m_{1v}{\nabla\mathcal{L}_{v}}}} \\\vdots \\{{m_{v\; 1}{\nabla\mathcal{L}_{1}}} + \ldots + {m_{vv}{\nabla\mathcal{L}_{v}}}} \\{\nabla m_{11\;}} \\\vdots \\{\nabla m_{v\; 1}} \\\vdots \\{\nabla m_{1v}} \\\vdots \\{\nabla m_{vv}}\end{bmatrix}}\end{matrix} & \left( {3.12\text{-}14} \right)\end{matrix}$

Process for Determining Hessian Rank of Dynamic Lagrangian

Information for determining Hessian rank of dynamic Lagrangian isavailable elsewhere herein.

Converting the Lagrangian to the Hamiltonian Via the Legendretransformation.

In our formulation the Lagrangian, L_(k) ^((T))(q, {dot over (q)}; ω),may be converted to the Hamiltonian using the Legendre transformation,so that

$\begin{matrix}\begin{matrix}{{H_{k}^{(T)}\left( {q,{p;\omega}} \right)} = {{\frac{\partial L_{k}^{(T)}}{\partial\overset{.}{q}}\overset{.}{q}} - {L_{k}^{(T)}\left( {q,{\overset{.}{q};\omega}} \right)}}} \\{= {{p^{T}\overset{.}{q}} - {L_{k}^{(T)}\left( {q,{\overset{.}{q};\omega}} \right)}}}\end{matrix} & \left( {3.12\text{-}15} \right)\end{matrix}$

Pareto Multi-Criteria Optimization Engine (PMOE)

Consider the problem of determining the relaxed Pareto optimal solutionto a given system query at a given time step. There are N decisionelements, k=1, . . . , N. A given decision element, DE_(k), has thefollowing associated parameters which are constituent to the ARB:

-   -   A generalized set of coordinates relevant to DE_(k), q.    -   A generalized set of linearly independent momenta {p_(a)} where        the index a refers the linearly independent momenta selected        from the canonical set p.    -   A set of control parameters co for hard a soft rules of the        system, where 0≦ω_(i)≦1.

The ARB has the following components which determine the constraints ofDE_(k):

-   -   The Hamiltonian which identifies the fundamental dynamics of the        system of the system for the k'th decision element denoted

H_(k) ^((o))(q,{p_(a)})  (3.13-1)

-   -   The summation of the first class constraints of the system,        which is

$\begin{matrix}{\sum\limits_{i}{\omega_{i}{f_{i}\left( {q^{()},\omega_{i}} \right)}}} & \left( {3.13\text{-}2} \right)\end{matrix}$

-   -   The summation of the second class constraints of the system        which is

$\begin{matrix}{\sum\limits_{i}{g_{i}\left( {q^{()},\omega_{i}} \right)}} & \left( {3.13\text{-}3} \right)\end{matrix}$

-   -   The Tellegen agent which is a function of the Hamiltonians of        the absolute rules of the other N−1 decision elements in the        system

H _(k) ^((A)) =F _(k) ^((A))(H ₁ ^((T)) , . . . ,H _(k−1) ^((T)) ,H_(k+1) ^((T)) , . . . ,H _(K) ^((T)))  (3.13-4)

-   -   The total Hamiltonian of the system is denoted H^((T)).    -   Approximations to the various Hamiltonian's are denoted Ĥ_(k)        ^((A)), Ĥ^((T)) and Ĥ_(k) ^((o)) for the Tellegen, total, and        DE-level Hamiltonians respectively.

System Initialization

Determining the relaxed Pareto optimal point of the system is a processwhich includes:

-   -   Initialization of N decision elements.    -   Synchronization through companionship of each of the N decision        elements with its respective Tellegen agent.

System Operation

Illustration SS shows how decision elements interact with the network,receive queries, and return results. In this example, the distributedsystem effectively implements an abstract classifier that has no realimplementation. The DE's receive sensor data from the network whichincludes new available information which may benefit classification. Theuser submits a query that is received by a DE which then returns aresult.

Gauge Systems in a Hamiltonian Domain

The time integral of the Lagrangian L(q, {dot over (q)}) is the actionS_(L) defined as

S _(L)=∫_(t) ₁ ^(t) ² L(q,{dot over (q)})dt

where

$\overset{.}{q} = {\frac{{q(t)}}{t}.}$

The Lagrangian conditions for stationarity are first that

$\begin{matrix}{{{\frac{\;}{t}L_{{\overset{.}{q}}^{(n)}}} - L_{q^{(n)}}} = 0} & \left( {3.14\text{-}1} \right)\end{matrix}$

where n=1, . . . , N,

${L_{{\overset{.}{q}}^{(n)}} = \frac{\partial L}{\partial{\overset{.}{q}}^{(n)}}},{and}$$L_{q^{(n)}} = {\frac{\partial L}{\partial q^{(n)}}.}$

And, secondarily

$\begin{matrix}{{\left\lbrack {\sum\limits_{n^{\prime} = 1}^{N}{\overset{¨}{q}}^{(n^{\prime})}} \right\rbrack L_{{\overset{.}{q}}^{(n)}{\overset{.}{q}}^{(n)}}} = {L_{q^{(n)}} - {{\overset{.}{q}}^{(n)}L_{{\overset{.}{q}}^{(n)}q^{(n)}}}}} & \left( {3.14\text{-}2} \right)\end{matrix}$

where

${\overset{¨}{q}}^{(n^{\prime})} = \frac{^{2}q^{(n^{\prime})}}{t^{2}}$and$L_{{\overset{.}{q}}^{(n)}q^{(n)}} = {\frac{\partial^{2}L}{\partial\left( {\overset{.}{q}}^{(n)} \right)^{2}}.}$

The generalized accelerations {umlaut over (q)}^((n)) are immediatelydetermined if L_({dot over (q)})(n)_({dot over (q)})(n) is invertible,or equivalently

det(L _({dot over (q)}(n){dot over (q)}(n))≠)0  (3.14-3)

for i=1, . . . , N. If for some n,det(L_({dot over (q)})(n)_({dot over (q)})(n))=0, the accelerationvector {umlaut over (q)}^((n)) will not be uniquely determined.

The departing point for the Hamiltonian approach is the definition ofconjugate momentum

p _(n) =L _({dot over (q)})(n)  (3.14-4)

where n=1, . . . , N. We will see that (3.14-3) is the condition ofnon-invertibility of

$L_{\overset{.}{q}\overset{.}{q}} = \begin{bmatrix}L_{{\overset{.}{q}}^{(1)}{\overset{.}{q}}^{(1)}} & \ldots & L_{{\overset{.}{q}}^{(n)}{\overset{.}{q}}^{(N)}} \\\vdots & ⋰ & \vdots \\L_{{\overset{.}{q}}^{(N)}{\overset{.}{q}}^{(1)}} & \ldots & L_{{\overset{.}{q}}^{(N)}{\overset{.}{q}}^{(N)}}\end{bmatrix}$

of the velocities of the functions of the coordinates q and momenta p.In other words, in this case, the momenta defined in (3.14-4) are notall independent. Define the relations that follow from (3.14-4) as

φ_(m)(q,p)  (3.14-5)

where m=1, . . . , M. Write (3.14-4) in vector notation as

p=L _({dot over (q)})(q,{dot over (q)}).

Then compatibility demands

φ_(m)(q,L _({dot over (q)})(q,{dot over (q)}))=0

is an identity with m=1, . . . , M.

Relations specified in (3.14-5) are called primary constraints. Forsimplicity let's assume that rank(L_({dot over (q)}{dot over (q)})) isconstant throughout the phase space, (q, {dot over (q)}), so that(3.14-5) defines a submanifold smoothly embedded in the phase space.This manifold is known as the primary constraint surface. Let

rank(L _({dot over (q)}{dot over (q)}))=N−M′  (3.14-6)

Then there are M′ independent constraints among (3.14-5) and the primaryconstraint surface is a phase space submanifold of dimension 2N−M′.

We do not assume that all the constraints are linearly independent sothat

M′≦M.  (3.14-7)

It follows from (3.14-5) that the inverse transformation from the p's tothe q's is multivalued. That is, given q, p that satisfies (3.14-5), theinverse image (q, that satisfies

$\begin{matrix}{p = \left( \frac{\partial L}{\partial\overset{.}{q}} \right)^{T}} & \left( {3.14\text{-}8} \right)\end{matrix}$

is not unique, since (3.14-8) defines a map from a 2N-dimensionalmanifold (q, {dot over (q)}) to the smaller (2N−M′)-dimensionalmanifold. Thus the inverse image of the points of (3.14-5) form amanifold of dimension M′.

Conditions on the Constraint Function

There exist many equivalent ways to represent a given surface by meansof equations of the form of (3.14-5). For example the surface p₁=0 canbe represented equivalently by p₁ ²=0, √{square root over (|p₁|)}=0, orredundantly by p₁=0 and p₁ ²=0. To use the Hamiltonian formalism, it isnecessary to impose some restrictions which the regularity conditionsfor the constraints.

Regularity Conditions

The (2N−M′)-dimensional constraint surface φ_(m)(q, p) should be coveredof open region: in each region the constraints can be split intoindependent constraints

{φ_(m′) |m′=1, . . . ,M′}.

Their Jacobian matrix

$\left\{ \frac{\partial\varphi_{m^{\prime}}}{{\partial p_{n}},q^{(n)}} \right\} = \begin{bmatrix}\frac{\partial\varphi_{1}}{{\partial p_{1}},q^{(1)}} & \ldots & \frac{\partial\varphi_{1}}{{\partial p_{n}},q^{(n)}} \\\vdots & ⋰ & \vdots \\\frac{\partial\varphi_{m^{\prime}}}{{\partial p_{1}},q^{(1)}} & \ldots & \frac{\partial\varphi_{m^{\prime}}}{{\partial p_{n}},q^{(n)}}\end{bmatrix}$

with m′=1, . . . , M′ and n=1, . . . , N, is of rank M′.

The dependent constraints φ_(m), m=M′+1, . . . , M of the other φ_(m′)=0

φ_(m)=0. Alternatively the condition on the Jacobian.

-   1. The function φ_(m′) can be taken locally as the first M′    coordinates of a new regular system in the vicinity of the    constraint surface or the differentials dφ₁, . . . , dφ_(M′) are    locally linearly independent:

dφ₁

. . .

dφ_(M′)≠0  (3.14-9)

-   2. The variations δφ_(m′) are of order ε for arbitrary variations    δq^((i)), δp_(i) of order ε (Dirac's approach).

Theorem 3.14.1. If a smooth, phase space function G vanishes on{φ_(m)=0} then

$\begin{matrix}{G = {\sum\limits_{m = 1}^{M}{g^{(m)}\varphi_{m}}}} & \left( {3.14\text{-}10} \right)\end{matrix}$

Proof: (local proof). Set φ_(m′), m′=1, . . . , M′ as coordinates(y_(m′),x_(α)) with y_(m′)=φ_(m′). In these coordinates G(0, x)=0 and

$\begin{matrix}\begin{matrix}{{G\left( {y,x} \right)} = {\int_{0}^{1}{\frac{\;}{t}{G\left( {{ty},x} \right)}\ {t}}}} \\{{\sum\limits_{m^{\prime} = 1}^{M^{\prime}}{y_{m^{\prime}}{\int_{0}^{1}{\frac{\partial\;}{\partial y_{m^{\prime}}}{G\left( {{ty},x} \right)}\ {t}}}}}} \\{{\sum\limits_{m^{\prime} = 1}^{M^{\prime}}{{g^{(m^{\prime})}\left( {y,x} \right)}{\varphi_{m^{\prime}}\left( {y,x} \right)}}}}\end{matrix} & \left( {3.14\text{-}11} \right) \\{{with}{{g^{(m^{\prime})}\left( {y,x} \right)} = {\int_{0}^{1}{\frac{\partial\;}{\partial y_{m^{\prime}}}{G\left( {{ty},x} \right)}\ {{t}.}}}}} & \;\end{matrix}$

Theorem 3.14.2. If the sum Σ(λ^((n))δq^((n))+μ_(n)δp_(n))=0 forarbitrary variations δq^((i)), δp_(i) tangent to the constraint surface{φ_(m)(q, p)=0|m=1, . . . , M}, then

$\begin{matrix}{\lambda^{(n)} = {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial q^{(n)}}}}} & \left( {3.14\text{-}12} \right) \\{\lambda_{n} = {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial q_{n}}}}} & \left( {3.14\text{-}13} \right)\end{matrix}$

Proof. The dimension of {φ_(m)} is 2N-M′. Thus the variations at a point(p, q) forms a 2N-M′ dimensional space

$\begin{matrix}{{\sum\limits_{n = 1}^{N}\left( {{\lambda^{(n)}\delta \; q^{(n)}} + {\mu_{n}\delta \; p_{n}}} \right)} = 0} & \left( {3.14\text{-}14} \right)\end{matrix}$

By the singularity assumption, there exists exactly M′ solutions to(3.14-14). Clearly, the gradients

$\left\{ \frac{\partial\varphi_{m^{\prime}}}{\partial q^{(n)}} \right\} \mspace{14mu} {and}\mspace{20mu} \left\{ \frac{\partial\varphi_{m^{\prime}}}{\partial p_{n}} \right\}$

are linearly independent. They are the basis for solutions to (3.14-14).

Note that in the presence of redundant constraints, the functionsu^((m)) exist but are not unique.

Canonical Hamiltonian

The Hamiltonian in canonical coordinates is

$\begin{matrix}{{H\left( {q,p} \right)} = {{\sum\limits_{n = 1}^{N}{{\overset{.}{q}}^{(n)}p_{n}}} - {L\left( {q,\overset{.}{q}} \right)}}} & \left( {3.14\text{-}15} \right)\end{matrix}$

The rate {dot over (q)} enters through the combination through conjugatemomenta defined for each coordinate

p _(n)(q,{dot over (q)})=L _({dot over (q)}) _((n)) (q,{dot over(q)})  (3.14-16)

This remarkable property is essential for the Hamiltonian approach. Itis verified by evaluating the change 6H involved by arbitraryindependent variations of position and velocities.

$\begin{matrix}\begin{matrix}{{\delta \; H} = {{\sum\limits_{n = 1}^{N}\left( {{{\overset{.}{q}}^{(n)}\delta \; p_{n}} + {\delta \; {\overset{.}{q}}^{(n)}p_{n}}} \right)} - {\delta \; L}}} \\{= {{\sum\limits_{n = 1}^{N}\left( {{{\overset{.}{q}}^{(n)}\delta \; p_{n}} + {\delta \; {\overset{.}{q}}^{(n)}p_{n}}} \right)} - {\sum\limits_{n = 1}^{N}\left( {{L_{q^{(n)}}\delta \; q^{(n)}} + {L_{{\overset{.}{q}}^{(n)}}\delta \; {\overset{.}{q}}^{(n)}}} \right)}}}\end{matrix} & \left( {3.14\text{-}17} \right)\end{matrix}$

Utilizing (3.14-16) in (3.14-17) yields

$\begin{matrix}{{\delta \; H} = {\sum\limits_{n = 1}^{N}\left( {{{\overset{.}{q}}^{(n)}\delta \; p_{n}} - {L_{q^{(n)}}\delta \; q^{(n)}}} \right)}} & \left( {3.14\text{-}18} \right)\end{matrix}$

The Hamiltonian defined by (3.14-15) is not unique as a function of p,q. This can be inferred from (3.14-18) by noticing that {δp_(n)|n=1, . .. , N} are not all independent. They are restricted to preserve theprimary constraints φ_(m)≈0 which are identities when the p's areexpressed as functions of q's via (3.14-16).

Using the definition of the differential in several variables applied toδH=δH({q^((n))}, {p_(n)}), (3.14-18) can be rewritten

$\begin{matrix}{{{\sum\limits_{n = 1}^{N}\left( {{\frac{\partial H}{\partial q^{(n)}}\delta \; q^{(n)}} + {\frac{\partial H}{\partial p_{n}}\delta \; p_{n}}} \right)} = {\sum\limits_{n = 1}^{N}\left( {{{\overset{.}{q}}^{(n)}\delta \; p_{n}} - {\delta \; q^{(n)}\frac{\partial L}{\partial q^{(n)}}}} \right)}}{or}{{{\sum\limits_{n = 1}^{N}{\left( {\frac{\partial H}{\partial q^{(n)}} + \frac{\partial H}{\partial q^{(n)}}} \right)\delta \; q^{(n)}}} + {\sum\limits_{n = 1}^{N}{\left( {\frac{\partial H}{\partial p_{n}} - {\overset{.}{q}}^{(n)}} \right)\delta \; p_{n}}}} = 0}} & \left( {3.14\text{-}19} \right)\end{matrix}$

From theorem 2 we then conclude for each n that.

$\begin{matrix}{{{\frac{\partial H}{\partial q^{(n)}} + \frac{\partial L}{\partial q^{(n)}}} = {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial q^{(n)}}\mspace{14mu} {and}}}}{{\frac{\partial H}{\partial p_{n}} - {\overset{.}{q}}^{(n)}} = {\sum\limits_{m = 1}^{M}{u^{(m)}{\frac{\partial\varphi_{m}}{\partial p_{n}}.}}}}} & \left( {3.14\text{-}20} \right)\end{matrix}$

So for each n:

$\begin{matrix}{{{\overset{.}{q}}^{(n)} = {\frac{\partial H}{\partial p_{n}} + {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial p_{n}}}}}},{n = 1},\ldots \;,{N\mspace{14mu} {and}}} & \left( {3.14\text{-}21} \right) \\{{{- \frac{\partial L}{\partial q^{(n)}}} = {\frac{\partial H}{\partial q^{(n)}} + {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial q^{(n)}}}}}},{n = 1},\ldots \;,{N.}} & \left( {3.14\text{-}22} \right)\end{matrix}$

Note that if the constraints are independent, the vectors

${\sum\limits_{n = 1}^{N}\frac{\partial\varphi_{m}}{\partial p_{n}}},$

m=1, . . . , M are also independent because of the regularity conditions(this is proved later). Hence no two sets of {u^((m))|m=1, . . . , M}can yield the same velocities via (3.14-21).

Thus, using

${\overset{.}{q}}^{(n)} = {\frac{\partial H}{\partial p_{n}} + {\sum\limits_{m = 1}^{M}{{u^{(m)}\left( {q,\overset{.}{q}} \right)}\frac{\partial\varphi_{m}}{\partial p_{n}}\left( {q,{p\left( {q,\overset{.}{q}} \right)}} \right)}}}$

we can find u^((m))(p,{dot over (q)}). If we define the transformationfrom (q,{dot over (q)}) to the manifold {φ_(m)(q, p)=0|m=1, . . . , M},from q, {dot over (q)}, u→q, p, u by

q=q, n=1, . . . ,N

p _(n) =L _(q) _((n)) (q,{dot over (q)}), n=1, . . . ,N−M′

u ^((m)) =u ^((m))(q,{dot over (q)}), m=1, . . . ,M′

We see that this transformation is invertible since one has from q, p,u→q, {dot over (q)}, u

q = q${\overset{.}{q}}^{(n)} = {\frac{\partial H}{\partial p_{n}} + {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial p_{n}}}}}$φ_(n)(q, p) = 0

Thus invertibility of the Legendre transformation when

det(L _({dot over (q)}{dot over (q)}))=0

can be regained at the prices of adding extra variables.

Action Principle of the Hamiltonian Form

With (3.14-21) and (3.14-22) we can rewrite (3.14-1) in the equivalentHamiltonian form

$\begin{matrix}{{{\overset{.}{q}}^{(n)} = {\frac{\partial H}{\partial p_{n}} + {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial p_{n}}}}}}{{\overset{.}{p}}_{n} = {{- \frac{\partial H}{\partial p_{n}}} - {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial p_{n}}}}}}{{{\varphi_{m}\left( {q,p} \right)} = 0},{m = 1},\ldots \;,M^{\prime}}} & \left( {3.14\text{-}23} \right)\end{matrix}$

The Hamiltonian Equations (3.14-23) can be derived from the followingvariational principle:

$\begin{matrix}{{\delta {\int_{t_{1}}^{t_{2}}\left\lbrack {{\sum\limits_{n = 1}^{N}{q^{(n)}p_{n}}} - H - {\sum\limits_{m = 1}^{M}{u^{(m)}\varphi_{m}}}} \right\rbrack}} = 0} & \left( {3.14\text{-}24} \right)\end{matrix}$

for arbitrary variations of δq^((n)), δp_(n), and δu^((m)) subject to

δq(t ₁)=δq(t ₂)=0

where the u^((m)) appear now as Lagrange multipliers enforcing theprimary constraints

φ_(m)(q,p)=0, m=1, . . . ,M.

Let F(p, q) be an arbitrary function of the canonical variables, then

$\begin{matrix}\begin{matrix}{\frac{F}{t} = {{\sum\limits_{n = 1}^{N}{\frac{\partial F}{\partial q^{(n)}}{\overset{.}{q}}_{n}}} + {\sum\limits_{n = 1}^{N}{\frac{\partial F}{\partial p_{n}}{\overset{.}{p}}_{n}}}}} \\{= {{\sum\limits_{n = 1}^{N}{\frac{\partial F}{\partial q^{(n)}}\left\lbrack {\frac{\partial H}{\partial p_{n}} + {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial p_{n}}}}} \right\rbrack}} +}} \\{{\sum\limits_{n = 1}^{N}{\frac{\partial F}{\partial p_{n}}\left\lbrack {{- \frac{\partial H}{\partial p_{n}}} - {\sum\limits_{m = 1}^{M}{u^{(m)}\frac{\partial\varphi_{m}}{\partial p_{n}}}}} \right\rbrack}}} \\{= {\left\lbrack {F,H} \right\rbrack + {\sum\limits_{m = 1}^{M}{u^{(m)}\left\lbrack {F,\varphi_{m}} \right\rbrack}}}}\end{matrix} & \left( {3.14\text{-}25} \right)\end{matrix}$

The equation (3.14-25) introduces the new binary operator [•,•] which isthe Poisson bracket and has the form

$\begin{matrix}\begin{matrix}{\left\lbrack {F,G} \right\rbrack = {\sum\limits_{n = 1}^{N}\left\lbrack {{\frac{\partial F}{\partial q^{(n)}}\frac{\partial G}{\partial p_{n}}} + {\frac{\partial F}{\partial p_{n}}\frac{\partial G}{\partial q^{(n)}}}} \right\rbrack}} \\{= {\sum\limits_{n = 1}^{N}\left\lbrack {{F_{q^{(n)}}G_{p_{n}}} + {F_{p_{n}}G_{q^{(n)}}}} \right\rbrack}}\end{matrix} & \left( {3.14\text{-}26} \right)\end{matrix}$

Secondary Constraints

The basic consistency condition is that the primary constraints bepreserved in time. So for

F(p,q)=φ_(m)(q,p)

we should have that {dot over (φ)}_(m)=0. {φ_(m)(q, p)=0}. So this means

$\begin{matrix}{{\left\lbrack {\varphi_{m},H} \right\rbrack + {\sum\limits_{m^{\prime} = 1}^{M^{\prime}}{u^{(m^{\prime})}\left\lbrack {\varphi_{m},\varphi_{m^{\prime}}} \right\rbrack}}} = 0} & \left( {3.14\text{-}27} \right)\end{matrix}$

This equation can either reduce to a relation independent of theu^((m′)), or, it may impose a restriction on the u's.

U=−{[φ _(m),φ_(m′)]}[φ_(m) ,H](q,p)  (3.14-28)

In the case (3.14-27) is independent of the u's (3.14-27) is called asecondary constraint. The fundamental difference of secondaryconstraints with respect to primary constraints is that primaryconstraints is that primary constraints are the consequence of thedefinition (3.14-8) while secondary constraints depend on the dynamics.

If X(q, p)=0 is an external constraint, we most impose a compatibilitycondition

$\begin{matrix}{{\left\lbrack {X,H} \right\rbrack + {\sum\limits_{m = 1}^{M^{\prime}}{u^{(m)}\left\lbrack {X,\varphi_{m}} \right\rbrack}}} = 0} & \left( {3.14\text{-}29} \right)\end{matrix}$

Next we need to test whether this constraint:

$\begin{matrix}{\mspace{79mu} {{\Phi \left( {p,q} \right)} = {{\left\lbrack {X,H} \right\rbrack + {\sum\limits_{m = 1}^{M^{\prime}}{u^{(m)}\left\lbrack {X,\varphi_{m}} \right\rbrack}}} = 0}}} & \left( {3.14\text{-}30} \right) \\{\mspace{79mu} {\text{?}{\text{?}\text{indicates text missing or illegible when filed}}}} & \left( {3.14\text{-}31} \right)\end{matrix}$

Implies new secondary constraints or whether it only restricts the u's.After the process is finished we are left with a number of secondaryconstraints which will be denoted by

φ_(k)=0, k=M+1, . . . ,M+K

where K is the total number of secondary constraints. In general, itwill be useful to denote all the constraints (primary and secondary) ina uniform way as

φ_(j)(q,p)=0, j=1, . . . ,M+K=J  (3.14-32)

We make the same regularity assumptions on the full set of constraints.

Weak and Strong Equations

Equation (3.14-32) can be written as

φ_(j)(•)≈0  (3.14-33)

To emphasize, the quantity φ_(j) is numerically restricted to be zerobut does not vanish throughout the space. What this is means is thatφ_(j) has non-zero Poisson brackets with the canonical variables.

Let F, G be functions that coincide on the manifold {φ_(j)≈0|j=1, . . ., J} are said the be weakly equal and denoted by F≈G. On the other hand,an equation that holds throughout the entire phase space and not just onthe submanifold {φ_(j)≈0} is called strong. Hence, by theorem 1

$\begin{matrix}{\left. {F \approx G}\Leftrightarrow{F - G} \right. = {\sum\limits_{j = 1}^{J}{{c^{(j)}\left( {p,q} \right)}\varphi_{j}}}} & \left( {3.14\text{-}34} \right)\end{matrix}$

Restrictions on the Lagrange Multipliers

Assume that we have found a complete set of constraints

$\begin{matrix}\left\{ {{\left. {\varphi_{j} \approx 0} \middle| j \right. = 1},\ldots \mspace{14mu},J} \right\} & \left( {3.14\text{-}35} \right) \\{{\left\lbrack {\varphi_{j},H} \right\rbrack + {\sum\limits_{m = 1}^{M}{u^{(m)}\left\lbrack {\varphi_{j},\varphi_{m}} \right\rbrack}}} \approx 0} & \left( {3.14\text{-}36} \right)\end{matrix}$

We consider (3.14-36) as a set of non-homogeneous linear equations withM≦J unknowns with coefficients that are functions of the q's and p's.

The general solution of (3.14-36) for each j is of the form

u ^((m)) =U ^((m)) +V ^((m)) , m=1, . . . ,M  (3.14-37)

with V^((m)) the solution of the homogeneous equation

$\begin{matrix}{{\sum\limits_{m = 1}^{M}{V^{(m)}\left\lbrack {\varphi_{j},\varphi_{m}} \right\rbrack}} \approx 0} & \left( {3.14\text{-}38} \right)\end{matrix}$

The most general solution of (3.14-38) is a linear combination oflinearly independent solutions of V_(α) ^((m)) where α=1, . . . , A withA≦M. Under the assumption that the matrix

$\begin{matrix}\begin{bmatrix}\left\lbrack {\varphi_{1},\varphi_{1}} \right\rbrack & \ldots & \left\lbrack {\varphi_{1},\varphi_{M}} \right\rbrack \\\vdots & ⋰ & \vdots \\\left\lbrack {\varphi_{J},\varphi_{1}} \right\rbrack & \ldots & \left\lbrack {\varphi_{J},\varphi_{M}} \right\rbrack\end{bmatrix} & \left( {3.14\text{-}39} \right)\end{matrix}$

is of constant rank, the number of independent solutions A is the samefor all p, q. Thus the general solution to (3.14-36) can be written as

$\begin{matrix}{{u^{(m)} \approx {U^{(m)} + {\sum\limits_{\alpha = 1}^{A}{v^{(\alpha)}V_{\alpha}^{(m)}}}}},{m = 1},\ldots \mspace{14mu},M} & \left( {3.14\text{-}40} \right)\end{matrix}$

Irreducible and Reducible Cases

If the equations {φ_(j)=0|j=1, . . . , J} are not independent, one saysthat the constraints are reducible. The system is irreducible when theconstraints are independent. However the separation of constraints intodependent and independent ones might be difficult to perform. It alsomay disturb invariance properties under some important symmetry. In somecases it may be impossible to separate irreducible from irreduciblecontexts. Reducible cases arise for example when the dynamicalcoordinates include p-form gauge fields.

Any irreducible set of constraints can always be replaced by a reducibleset by introducing constraints of the ones already at hand. Theformalism should be invariant under such replacements.

Total Hamiltonian

We now discuss details of the dynamic equation (3.14-25)

$\begin{matrix}{\overset{.}{F} \approx \left\lbrack {F,{H^{\prime} + {\sum\limits_{\alpha = 1}^{A}{v^{(\alpha)}\varphi_{\alpha}}}}} \right\rbrack} & \left( {3.14\text{-}41} \right)\end{matrix}$

where from (3.14-40)

$\begin{matrix}{{H^{\prime} = {H + {\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}}}}{and}{{\varphi_{\alpha} = {\sum\limits_{m = 1}^{M}{V_{\alpha}^{(m)}\varphi_{m}}}},{\alpha = 1},\ldots \mspace{14mu},A}} & \left( {3.14\text{-}42} \right)\end{matrix}$

This is the result of theorem 3 (see below).

Theorem 3.

$\begin{matrix}{\left\lbrack {F,{\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}}} \right\rbrack \simeq {\sum\limits_{m = 1}^{M}{U^{(m)}\left\lbrack {F,\varphi_{m}} \right\rbrack}}} & \left( {3.14\text{-}43} \right) \\{\left\lbrack {F,{\sum\limits_{\alpha = 1}^{A}{V_{\alpha}^{(m)}\varphi_{m}}}} \right\rbrack \simeq {\sum\limits_{\alpha = 1}^{A}{V_{\alpha}^{(m)}\left\lbrack {F,\varphi_{m}} \right\rbrack}}} & \left( {3.14\text{-}44} \right) \\\begin{matrix}\begin{matrix}{\left\lbrack {F,{\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}}} \right\rbrack = {\sum\limits_{i = 1}^{N}\left\{ {{\frac{\partial F}{\partial q^{(i)}}\frac{\partial}{\partial p_{i}}{\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}}} -} \right.}} \\\left. {\frac{\partial F}{\partial p_{i}}\frac{\partial}{\partial q^{(i)}}{\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}}} \right\} \\{= {\sum\limits_{i = 1}^{N}\left\{ {\frac{\partial F}{\partial q^{(i)}}\left\lbrack {{\sum\limits_{m = 1}^{M}{\frac{\partial U^{(m)}}{\partial p_{i}}\varphi_{m}}} +} \right.} \right.}} \\{\left. \left. {\sum\limits_{m = 1}^{M}{U^{(m)}\frac{\partial\varphi_{m}}{\partial p_{i}}}} \right\rbrack \right\} -} \\{{\sum\limits_{i = 1}^{N}\left\{ {\frac{\partial F}{\partial p_{i}}\left\lbrack {{\sum\limits_{m = 1}^{M}{\frac{\partial U^{(m)}}{\partial q^{(i)}}\varphi_{m}}} +} \right.} \right.}} \\\left. \left. {\sum\limits_{m = 1}^{M}{U^{(m)}\frac{\partial\varphi_{m}}{\partial q^{(i)}}}} \right\rbrack \right\} \\{= {\sum\limits_{m = 1}^{M}\left\{ {{\left\lbrack {F,U^{(m)}} \right\rbrack \varphi_{m}} + {U^{(m)}\left\lbrack {F,\varphi_{m}} \right\rbrack}} \right\}}}\end{matrix} \\{So} \\{{\left\lbrack {F,{\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}}} \right\rbrack - {\sum\limits_{m = 1}^{M}{U^{(m)}\left\lbrack {F,\varphi_{m}} \right\rbrack}}} = {\sum\limits_{m = 1}^{M}{\left\lbrack {F,U^{(m)}} \right\rbrack \varphi_{m}}}}\end{matrix} & \left( {3.14\text{-}45} \right)\end{matrix}$

Proof.

and from (3.14-34) in (3.14-45), (3.14-43) follows. By a similar processwe show (3.14-44). We now prove the validity of (3.14-41).

Theorem 4. Let F(q,p) be a regular function, then F(p,q) propagates intime according to the approximate equation (3.14-41).

Proof. From (3.14-25),

$\begin{matrix}{\frac{F}{t} = {\left\lbrack {F,H} \right\rbrack + {\sum\limits_{m = 1}^{M}{{u^{(m)}\left\lbrack {F,\varphi_{m}} \right\rbrack}.}}}} & \left( {3.14\text{-}46} \right)\end{matrix}$

From (3.14-40) into (3.14-46) we obtain,

$\begin{matrix}{\mspace{79mu} {{\frac{F}{t} \approx {\left\lbrack {F,H} \right\rbrack + {\sum\limits_{m = 1}^{M}{\left\{ {U^{(m)} + {\sum\limits_{\alpha = 1}^{A}{v^{(\alpha)}V_{\alpha}^{(m)}}}} \right\} \left\lbrack {F,\varphi_{m}} \right\rbrack}}}}\mspace{20mu} {or}{\frac{F}{t} \approx {\left\lbrack {F,H} \right\rbrack + {\sum\limits_{m = 1}^{M}{U^{(m)}\left\lbrack {F,\varphi_{m}} \right\rbrack}} + {\sum\limits_{m = 1}^{M}{\sum\limits_{\alpha = 1}^{A}{v^{(\alpha)}{V_{\alpha}^{(m)}\left\lbrack {F,\varphi_{m}} \right\rbrack}}}}}}}} & \left( {3.14\text{-}47} \right)\end{matrix}$

Thus from (3.14-43) and (3.14-44) of theorem 3, we get

$\begin{matrix}\begin{matrix}{\frac{F}{t} \approx {\left\lbrack {F,H} \right\rbrack + {\sum\limits_{m = 1}^{M}\left\lbrack {F,{U^{(m)}\varphi_{m}}} \right\rbrack} +}} \\{{\sum\limits_{\alpha = 1}^{A}{v^{(\alpha)}\left\lbrack {F,{\sum\limits_{m = 1}^{M}{V_{\alpha}^{(m)}\varphi_{m}}}} \right\rbrack}}} \\{\approx \left\lbrack {F,{H + {\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}} + {\sum\limits_{\alpha = 1}^{A}{v^{(\alpha)}{\sum\limits_{m = 1}^{M}{V_{\alpha}^{(m)}\varphi_{m}}}}}}} \right\rbrack} \\{\approx \left\lbrack {F,{H^{\prime} + {\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}} + {\sum\limits_{\alpha = 1}^{A}{v^{(\alpha)}\varphi_{\alpha}}}}} \right\rbrack}\end{matrix} & \left( {3.14\text{-}48} \right) \\{with} & \; \\{H^{\prime} = {H + {\sum\limits_{m = 1}^{M}{U^{(m)}\varphi_{m}}}}} & \left( {3.14\text{-}49} \right) \\{\varphi_{\alpha} = {\sum\limits_{m = 1}^{M}{V_{\alpha}^{(m)}\varphi_{m}}}} & \left( {3.14\text{-}50} \right)\end{matrix}$

Now define

$\begin{matrix}{H_{T} = {H^{\prime} + {\sum\limits_{\alpha = 1}^{A}\; {V^{(\alpha)}{\varphi_{\alpha}.}}}}} & \left( {3.14\text{-}51} \right)\end{matrix}$

So we obtain

$\begin{matrix}{\frac{F}{t} \approx \left\lbrack {F,H_{T}} \right\rbrack} & \left( {3.14\text{-}52} \right)\end{matrix}$

First and Second Class Functions

The distinction between primary and secondary constraints is of littleimportance.

We now consider a fundamental classification. It depends on the conceptof first class and second class functions.

Definition 1. A function F(q, p) is said to be first class if itsPoisson bracket with every constraint vanishes weakly, [F,φ_(j)]≈0, j=1,. . . , J. A function of the canonical variables that is not first classis called second class. Thus F is second class if [F, φ_(k)]≈0 for atleast one k, k=1, . . . , M.

Theorem 5. If F and G are first class functions, then their Poissonbracket is also a first class function.

Proof: By Hypothesis,

$\begin{matrix}{\left\lbrack {F,\varphi_{j}} \right\rbrack = {\sum\limits_{k = 1}^{M}\; {f_{j}^{(k)}\varphi_{k}}}} & \left( {3.14\text{-}53} \right) \\{\left\lbrack {G,\varphi_{j}} \right\rbrack = {\sum\limits_{l = 1}^{M}\; {g_{j}^{(l)}\varphi_{l}}}} & \left( {3.14\text{-}54} \right)\end{matrix}$

Applying the Jacobi identity, we get

$\begin{matrix}{\left\lbrack {\left\lbrack {F,G} \right\rbrack,\varphi_{j}} \right\rbrack  =} & {{\left\lbrack {F,\left\lbrack {G,\varphi_{j}} \right\rbrack} \right\rbrack - \left\lbrack {G,\left\lbrack {F,\varphi_{j}} \right\rbrack} \right\rbrack}} \\{=} & {{\left\lbrack {F,{\sum\limits_{l = 1}^{M}\; {g_{j}^{(l)}\varphi_{l}}}} \right\rbrack - \left\lbrack {G,{\sum\limits_{k = 1}^{M}\; {f_{j}^{(k)}\varphi_{k}}}} \right\rbrack}} \\{=} & {{{\sum\limits_{i}\left\{ {{\frac{\partial F}{\partial q^{(i)}}\frac{\partial}{\partial p_{i}}{\sum\limits_{l = 1}^{M}\; {g_{j}^{(l)}\varphi_{l}}}} - {\frac{\partial F}{\partial p_{i}}\frac{\partial}{\partial q^{(i)}}{\sum\limits_{l = 1}^{M}\; {g_{j}^{(l)}\varphi_{l}}}}} \right\}} -}} \\ & {{\sum\limits_{n}\left\{ {{\frac{\partial G}{\partial q^{(n)}}\frac{\partial}{\partial p_{n}}{\sum\limits_{k = 1}^{M}\; {f_{j}^{(k)}\varphi_{k}}}} - {\frac{\partial G}{\partial p_{n}}\frac{\partial}{\partial q^{(n)}}{\sum\limits_{k = 1}^{M}\; {f_{j}^{(k)}\varphi_{k}}}}} \right\}}} \\{=} & {{{\sum\limits_{i}\left\{ {{\frac{\partial F}{\partial q^{(i)}}{\sum\limits_{l = 1}^{M}\; \left\{ {{\frac{\partial g_{j}^{(l)}}{\partial p_{i}}\varphi_{l}} + {g_{j}^{(l)}\frac{\partial\varphi_{l}}{\partial p_{i}}}} \right\}}} - {\frac{\partial F}{\partial p_{i}}{\sum\limits_{l = 1}^{M}\; \left\{ {{\frac{\partial g_{j}^{(l)}}{\partial q^{(i)}}\varphi_{l}} + {g_{j}^{(l)}\frac{\partial\varphi_{l}}{\partial q^{(i)}}}} \right\}}}} \right\}} -}} \\ & {{\sum\limits_{n}\left\{ {{\frac{\partial G}{\partial q^{(n)}}{\sum\limits_{k = 1}^{M}\; \left\{ {{\frac{\partial f_{j}^{(k)}}{\partial p_{n}}\varphi_{k}} + {f_{j}^{(k)}\frac{\partial\varphi_{k}}{\partial p_{n}}}} \right\}}} - {\frac{\partial G}{\partial p_{n}}{\sum\limits_{k = 1}^{M}\; \left\{ {{\frac{\partial f_{j}^{(k)}}{\partial q^{(n)}}\varphi_{k}} + {f_{j}^{(k)}\frac{\partial\varphi_{k}}{\partial q^{(n)}}}} \right\}}}} \right\}}} \\{=} & {{{\sum\limits_{l = 1}^{M}\; \left\{ {{\varphi_{l}{\sum\limits_{i}\left\{ {{\frac{\partial F}{\partial q^{(i)}}\frac{\partial g_{j}^{(l)}}{\partial p_{i}}} - {\frac{\partial F}{\partial p_{i}}\frac{\partial g_{j}^{(l)}}{\partial q^{(i)}}}} \right\}}} + {g_{j}^{(l)}{\sum\limits_{i}\left\{ {{\frac{\partial F}{\partial q^{(i)}}\frac{\partial\varphi_{l}}{\partial q^{(i)}}} - {\frac{\partial F}{\partial p_{i}}\frac{\partial\varphi_{l}}{\partial p_{i}}}} \right\}}}} \right\}} -}} \\ & {{\sum\limits_{k = 1}^{M}\; \left\{ {{\varphi_{k}{\sum\limits_{n}\left\{ {{\frac{\partial G}{\partial q^{(n)}}\frac{\partial f_{j}^{(k)}}{\partial p_{n}}} - {\frac{\partial G}{\partial p_{n}}\frac{\partial f_{j}^{(k)}}{\partial q^{(n)}}}} \right\}}} +} \right.}} \\ & \left. {f_{j}^{(k)}{\sum\limits_{n}\left\{ {{\frac{\partial G}{\partial q^{(n)}}\frac{\partial\varphi_{k}}{\partial p_{n}}} - {\frac{\partial G}{\partial p_{n}}\frac{\partial\varphi_{k}}{\partial q^{(n)}}}} \right\}}} \right\} \\{=} & {{{\sum\limits_{l = 1}^{M}\; \left\{ {{\varphi_{l}\left\lbrack {F,g_{j}^{(l)}} \right\rbrack} + {g_{j}^{(l)}\left\lbrack {F,\varphi_{l}} \right\rbrack}} \right\}} - {\sum\limits_{k = 1}^{M}\; \left\{ {{\varphi_{k}\left\lbrack {G,f_{j}^{(k)}} \right\rbrack} + {f_{j}^{(k)}\left\lbrack {G,\varphi_{k}} \right\rbrack}} \right\}}}} \\{=} & {{{\sum\limits_{l = 1}^{M}\; {\left\lbrack {F,g_{j}^{(l)}} \right\rbrack \varphi_{l}}} - {\sum\limits_{k = 1}^{M}\; {\left\lbrack {G,f_{j}^{(k)}} \right\rbrack \varphi_{k}}} + {\sum\limits_{l^{\prime} = 1}^{M}\; {\left\{ {\sum\limits_{l = 1}^{M}\; {g_{j}^{(l)}f_{l}^{(l^{\prime})}}} \right\} \varphi_{l^{\prime}}}} - {\sum\limits_{k^{\prime} = 1}^{M}\; {\left\{ {\sum\limits_{k = 1}^{M}\; {f_{j}^{(k)}g_{k^{\prime}}^{k}}} \right\} \varphi_{k^{\prime}}}}}} \\{\approx} & {0}\end{matrix}$

We now use theorem 5 to show the following.

Theorem 6. H′ defined by (3.14-49) and φ_(α) defined by (3.14-50) arefirst class functions.

Proof: This follows directly from (3.14-36) and (3.14-38).

We learn from theorem 6 that the total Hamiltonian defined by (3.14-51)is the sum of the first class Hamiltonian H′ and the first class primaryconstraints φ_(α) multiplied by arbitrary coefficients.

First Class Constraints as Generators of Gauge Transformations

Gauge transformations are transformations that do not change thephysical state.

The presence of arbitrary functions of time v^((α)), α=1, . . . , A inthe total Hamiltonian, H_(T) (see (3.14-51)) imply that not all the q'sand p's are observable given a set of q's and p's where the state of thephysical system is uniquely determined. However the converse is nottrue: there is more than one set of values of the canonical variablesthat defines a state. To illustrate this, we see that if we give aninitial set of values of physical state at time t, we expect theequations of motion to fully determine the state at other times. Thusany ambiguity in the value of the canonical variables at t₂≠t₁ should beirrelevant from the physical point of view.

A Derivation Example

We propose here an alternate formulation of Dirac's formalism.

Primary Constraints

Recall that the momenta, canonically conjugate to the generalized“coordinates” q^((j)), j=1, . . . , N is given by

$\begin{matrix}{{p_{j} = \frac{\partial{L\left( {q,\overset{.}{q}} \right)}}{\partial{\overset{.}{q}}^{(j)}}},{j = 1},\ldots,{N.}} & \left( {E\text{--}1} \right)\end{matrix}$

For non-singular systems the equations allows us to express {dot over(q)}^((j)), j=1, . . . , N in terms of the canonical variables,

{dot over (q)} ^((i)) =f _(i)(q,p), i=1, . . . ,N  (E--2)

By performing a Legendre transformation

${H_{c}\left( {p,q} \right)} = {{\sum\limits_{i = 1}^{N}\; {p_{i}{f\left( {q,p} \right)}}} + {L\left( {q,{f\left( {p,q} \right)}} \right)}}$

We obtain the Hamiltonian of the system H_(c). And from this function weobtain the standard equations of motion of the system.

$\begin{matrix}{{\overset{.}{q} = \frac{\partial H_{c}}{\partial p}}{\overset{.}{p} = {- \frac{\partial H_{c}}{\partial q}}}} & \left( {E\text{--}3} \right)\end{matrix}$

For (E--2) to be well-defined we need to have the Hessian W of satisfy

det W≠0  (E--4)

In this case the accelerations {umlaut over (q)}^((i)) are uniquelydetermined by the q^((j)) and {dot over (q)}^((i)).

When det W≠0, the Hamiltonian equations of motion do not take thestandard form, and we speak of a singular Lagrangian. For illustrationpurposes, consider a Lagrangian of the form

$\begin{matrix}{{L\left( {q,\overset{.}{q}} \right)} = {{\frac{1}{2}{\sum\limits_{i = 1}^{N}\; {\sum\limits_{j = 1}^{N}\; {{W_{ij}(q)}{\overset{.}{q}}^{(i)}{\overset{.}{q}}^{(j)}}}}} + {\sum\limits_{i = 1}^{N}\; {{\eta_{i}(q)}{\overset{.}{q}}^{(i)}}} - {V(q)}}} & \left( {E\text{--}5} \right)\end{matrix}$

with W a symmetric matrix. From (E--1), the canonical momentum for(E--5) is given by

$\begin{matrix}{{p_{i} = {{\frac{1}{2}{\sum\limits_{j = 1}^{N}\; {{W_{ij}(q)}{\overset{.}{q}}^{(j)}}}} + {\eta_{i}(q)}}},{i = 1},\ldots,{n.}} & \left( {E\text{--}6} \right)\end{matrix}$

If W is singular of rank R_(W), then it possesses N—R_(W) eigenvectorswith corresponding zero eigenvalues. Then for eigenvectors v_(j) ^((α))

${{\sum\limits_{j = 1}^{N}\; {{W_{ij}(q)}{v_{j}^{(\alpha)}(q)}}} = 0},{\alpha = 1},\ldots,{N - R_{W}}$

So pre-multiplying (E--6) by v_(j) ^((α)) and summing over i we get

$\begin{matrix}{\begin{matrix}{{\sum\limits_{i = 1}^{N}\; {{v_{i}^{(\alpha)}(q)}p_{i}}} = {\sum\limits_{i = 1}^{N}\; \left\lbrack {{\sum\limits_{j = 1}^{N}\; \left( {{v_{i}^{(\alpha)}(q)}{W_{ij}(q)}{\overset{.}{q}}^{(j)}} \right)} + {{v_{i}^{(\alpha)}(q)}{\eta_{i}(q)}}} \right\rbrack}} \\{{= {\sum\limits_{i = 1}^{N}\; {{\nu_{i}^{(\alpha)}(q)}{\eta_{i}(q)}}}},{\alpha = 1},\ldots,{N - R_{W}}}\end{matrix}{So}{{{\sum\limits_{i = 1}^{N}\; {{v_{i}^{(\alpha)}(q)}\left( {p_{i} - {\eta_{i}(q)}} \right)}} = 0},{\alpha = 1},\ldots,{N - {R_{W}.}}}} & \left( {E\text{--}7} \right)\end{matrix}$

Let {p_(α)}, α=1, . . . , N—R_(W), denote the linearly dependentelements of p. Let {p_(α)}, a=1, . . . , R_(a) be the momenta satisfying(E--1). Then the constraint equations are of the form

$\begin{matrix}{{{{{\sum\limits_{\beta = 1}^{N - R_{W}}\; {{N_{\alpha\beta}(q)}p_{\beta}}} - {F_{\alpha}\left( {q,\left\{ p_{a} \right\}} \right)}} = 0},{\alpha = 1},\ldots,{N - R_{W}}}{{M_{\alpha\beta}(q)} = v_{\beta}^{(\alpha)}}{and}} & \left( {E\text{--}8} \right) \\{{F_{\alpha}\left( {q,\left\{ p_{\beta} \right\}} \right)} = {{\sum\limits_{i = 1}^{N}\; {{v_{i}^{(\alpha)}(q)}{\eta_{i}(q)}}} + {\sum\limits_{b = 1}^{R_{W}}\; {{v_{b}^{(\alpha)}(q)}p_{b}}}}} & \left( {E\text{--}9} \right)\end{matrix}$

The matrix {M_(αβ)} is necessarily invertible because otherwise M wouldpossess eigenvectors with zero eigenvalues, implying existence ofadditional constraints.

Note that (E--8) can be written as

$\begin{matrix}{{{{p_{\alpha} - {g_{\alpha}\left( {q,\left\{ p_{a} \right\}} \right)}} = 0},{\alpha = 1},\ldots,{N - R_{W}}}{with}{{g_{\alpha}\left( {q,\left\{ p_{a} \right\}} \right)} = {\sum\limits_{\beta = 1}^{N - R_{W}}\; {M_{\alpha\beta}^{- 1}{F_{\beta}\left( {q,\left\{ p_{a} \right\}} \right)}}}}} & \left( {E\text{--}10} \right)\end{matrix}$

with dim{p_(a)}=R_(W). So we can define,

φ_(α)(q,p)=p _(α) −g _(α)(q,{p _(a)})=0, α=1, . . . ,N—R _(W)  (E--11)

In Dirac's terminology, constraints of the form of (E--11) are referredto as primary constraints. Although the derivation above is based on aLagrangian, quadratic in the velocity terms, it is generally valid forLagrangians which depend on q and {dot over (q)} but not on higherderivatives.

Note: Primary constraints follow exclusively from the definition ofcanonical momenta.

The derivation above is valid for general Lagrangians and their Hessian.Let's assume {W_(ij)(q, {dot over (a)})} is the Hessian of a givenLagrangian L. Let {W_(ab)|a, b=1, . . . , R_(W)} be the largestsub-matrix of {W_(ij)} with suitable rearrangement if necessary. We thensolve (E--1) for R_(W) velocities {dot over (q)}^((a)) in terms of{q^((i))|i=1, . . . , n}, {p_(a)|a=1, . . . R_(W)} and {q^((α))|α=1, . .. , N−R_(W)}. That is

{dot over (q)} ^((a)) =f _(a)(q,{p _(b) },{{dot over (a)}^((β))})  (E--12)

with a, b=1, . . . , R_(W) and β=R_(W)+1, . . . , N.

Inserting these relations into (E--1), we get relations of the form

p _(j) =h _(j)(q,{p _(a) },{{dot over (q)} ^((α))})  (E--13)

with a, j=1, . . . , R_(W) and α=R_(W)+1, . . . , N. This relationreduces to an identity by construction. The remaining equations are ofthe form

p _(α) =h _(α)(q,{p _(a) },{{dot over (q)} ^((β))})  (E--14)

with α=1, . . . , N−R_(W). However, the right hand side cannot depend on{{dot over (q)}^((β))} since otherwise we could express more velocitiesin terms of the momenta of the coordinates of the momenta and theremaining velocities.

Hamiltonian Equations of Motion for Constrained Systems

Theorem 3.16.1. In the space Γ_(p) define by Γ_(p)={φ_(α)(p, q)|α=1, . .. , N−R_(W)} where φ_(α) is defined as (E--11). The Hamiltonian is onlya function of {q^((i))|i=1, . . . , N} and momenta {p_(a)|a=1, . . . ,R_(W)} and does not depend on {{dot over (q)}^((α))|α=1, . . . ,N−R_(W)}

Proof. On Γ_(p) the Hamiltonian is given by

$\begin{matrix}{H_{o} = {\left. H_{c} \right|_{\Gamma_{p}} = {{\sum\limits_{a = 1}^{R_{W}}\; {p_{a}f_{a}}} - {\sum\limits_{\alpha = 1}^{N - R_{W}}\; {g_{\alpha}{\overset{.}{q}}^{(\alpha)}}} - {L\left( {q,\left\{ f_{b} \right\},\left\{ {\overset{.}{q}}^{(\beta)} \right\}} \right)}}}} & \left( {E\text{--}15} \right)\end{matrix}$

where f_(a), a=1, . . . , N−R_(W) is given by (E--12) and g_(α), α=1, .. . , R_(W) is given by (E--10). We want to show that H_(o) does notdepend on {dot over (q)}^((β)), β=1, . . . , N−R_(W). We compute

                                        (E–16) $\begin{matrix}{\frac{\partial H_{o}}{\partial{\overset{.}{q}}^{(\beta)}} = \left. {{\sum\limits_{a = 1}^{R_{W}}\; {p_{a}\frac{\partial f_{a}}{\partial{\overset{.}{q}}^{(\beta)}}}} - g_{\beta} - {\sum\limits_{a = 1}^{R_{W}}\; \frac{\partial L}{\partial{\overset{.}{q}}^{(a)}}}} \middle| {}_{{\overset{.}{q}}^{(a)} = f_{a}}{\frac{\partial f_{a}}{\partial{\overset{.}{q}}^{(\beta)}} - \frac{\partial L}{\partial{\overset{.}{q}}^{(\beta)}}} \right|_{{\overset{.}{q}}^{(a)} = f_{a}}} \\{= \left. {{\sum\limits_{\alpha = 1}^{R_{w}}\; {\left( \left. {p_{\alpha} - \frac{\partial L}{\partial{\overset{.}{q}}^{(\alpha)}}} \right|_{{\overset{.}{q}}^{(\alpha)} = f_{\alpha}} \right)\frac{\partial f_{\alpha}}{\partial{\overset{.}{q}}^{(\beta)}}}} - g_{\beta} - \frac{\partial L}{\partial{\overset{.}{q}}^{(\beta)}}} \right|_{{\overset{.}{q}}^{(\alpha)} = f_{\alpha}}}\end{matrix}$

Since by definition

${p_{a} = \frac{\partial L}{\partial{\overset{.}{q}}^{(a)}}},{a = 1},\ldots \mspace{11mu},R_{W}$

And from (E--11)

$\begin{matrix}{{g_{\beta} = {p_{\beta} = {{\frac{\partial L}{\partial{\overset{.}{q}}^{(\beta)}}_{{\overset{.}{q}}_{a} = f_{a}}{{.{So}}\mspace{14mu} \frac{\partial H_{o}}{\partial{\overset{.}{q}}^{(\beta)}}}} = 0}}},{\beta = 1},\ldots \mspace{11mu},{N - {R_{W}.}}} & \left( {E\text{-}17} \right)\end{matrix}$

and therefore

H _(o)(q,{p _(a) },{{dot over (q)} ^((a))})=H _(o)(q,{p _(a)}).

Theorem 3.16.2. In the presence of primary constraints (E--11), theHamilton equations of motion are given by

$\begin{matrix}{{{{\overset{.}{q}}^{(i)} = {\frac{\partial H_{o}}{\partial p_{i}} + {\sum\limits_{\beta = 1}^{N}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial\varphi_{\beta}}{\partial p_{i}}}}}},{i = 1},\ldots \mspace{11mu},N}{{{\overset{.}{p}}_{i} = {{- \frac{\partial H_{o}}{\partial q^{(i)}}} + {\sum\limits_{\beta = 1}^{n}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial\varphi_{\beta}}{\partial q^{(i)}}}}}},{i = 1},\ldots \mspace{11mu},N}{{{\varphi_{\alpha}\left( {p,q} \right)} = 0},{\alpha = 1},\ldots \mspace{11mu},{N - R_{W}}}} & \left( {E\text{-}18} \right)\end{matrix}$

where {dot over (q)}^((β)) are a priori underdetermined velocities.

Proof: From (E--15) we obtain and the application of Theorem 3.16.1

$\begin{matrix}\begin{matrix}{\frac{\partial H_{o}}{\partial p_{a}} = {f_{a} + {\sum\limits_{b = 1}^{R_{W}}\; {p_{b}\frac{\partial f_{b}}{\partial p_{a}}}} + {\sum\limits_{\beta = 1}^{N - R_{W}}\; {\frac{\partial g_{\beta}}{\partial p_{a}}{\overset{.}{q}}^{(\beta)}}} - {\sum\limits_{b = 1}^{R_{W}}\; {\frac{\partial L}{\partial{\overset{.}{q}}^{(b)}}\frac{\partial f_{b}}{\partial p_{a}}}}}} \\{= {{\overset{.}{q}}^{(a)} + {\sum\limits_{b = 1}^{R_{W}}\; {\left( {p_{b} - \frac{\partial L}{\partial{\overset{.}{q}}^{(b)}}} \right)\frac{\partial f_{b}}{\partial p_{a}}}} + {\sum\limits_{\beta = 1}^{N - R_{W}}\; {\frac{\partial g_{\beta}}{\partial p_{a}}{\overset{.}{q}}^{(\beta)}}}}} \\{= {{\overset{.}{q}}^{(a)} + {\sum\limits_{\beta = 1}^{N - R_{W}}\; {\frac{\partial g_{\beta}}{\partial p_{a}}{\overset{.}{q}}^{(\beta)}}}}}\end{matrix} & \left( {E\text{-}19} \right)\end{matrix}$

with a=1, . . . , n−R_(W). Further

$\begin{matrix}\begin{matrix}{\frac{\partial H_{o}}{\partial q^{(i)}} = {{{\sum\limits_{b = 1}^{R_{W}}\; {p_{b}\frac{\partial f_{b}}{\partial q^{(i)}}}} + {\sum\limits_{\beta = 1}^{N - R_{W}}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial g_{\beta}}{\partial q^{(i)}}}} - \frac{\partial L}{\partial q^{(i)}}}_{{\overset{.}{q}}_{a} = f_{a}} -}} \\{{{\sum\limits_{b = 1}^{R_{W}}\; \frac{\partial L}{\partial{\overset{.}{q}}_{b}}}_{{\overset{.}{q}}_{b} = f_{b}}\frac{\partial f_{b}}{\partial q^{(i)}}}} \\{= {{\sum\limits_{b = 1}^{R_{W}}\; {\left( {{p_{b} - \frac{\partial L}{\partial{\overset{.}{q}}^{(b)}}}_{{\overset{.}{q}}^{(b)} = f_{b}}} \right)\frac{\partial f_{b}}{\partial q^{(i)}}}} +}} \\{{{{\sum\limits_{\beta = 1}^{N - R_{W}}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial g_{\beta}}{\partial q^{\;^{(i)}}}}} - \frac{\partial L}{\partial q^{(i)}}}_{{\overset{.}{q}}^{(a)} = f_{a}}}} \\{= {{{\sum\limits_{\beta = 1}^{N - R_{W}}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial g_{\beta}}{\partial q^{(i)}}}} - \frac{\partial L}{\partial q^{(i)}}}_{{\overset{.}{q}}^{(a)} = f_{a}}}} \\{= {{{\sum\limits_{\beta = 1}^{N - R_{W}}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial g_{\beta}}{\partial q^{(i)}}}} - {\frac{}{t}\left( \frac{\partial L}{\partial{\overset{.}{q}}^{(i)}} \right)}}_{{\overset{.}{q}}^{(a)} = f_{a}}}}\end{matrix} & \left( {E\text{-}20} \right)\end{matrix}$

from (add reference).

$\begin{matrix}{\frac{\partial H_{o}}{\partial q^{(i)}} = {{- {\overset{.}{p}}_{i}} + {\sum\limits_{\beta = 1}^{N - R_{W}}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial g_{\beta}}{\partial q^{(i)}}}}}} & \left( {E\text{-}21} \right)\end{matrix}$

From (E--19) and (E--20) we get:

$\begin{matrix}{{{{\overset{.}{q}}^{(a)} = {\frac{\partial H_{o}}{\partial p_{a}} - {\sum\limits_{\beta = 1}^{N - R_{W}}\; {\frac{\partial g_{\beta}}{\partial p_{a}}{\overset{.}{q}}^{(\beta)}}}}},{a = 1},\ldots \mspace{11mu},R_{W}}{{{\overset{.}{p}}_{i} = {{- \frac{\partial H_{o}}{\partial q^{(i)}}} + {\sum\limits_{\beta = 1}^{n - R_{W}}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial g_{\beta}}{\partial q^{(i)}}}}}},{i = 1},\ldots \mspace{11mu},N}} & \left( {E\text{-}22} \right)\end{matrix}$

Since

$\frac{\partial H_{o}}{\partial p_{a}} = {{0\mspace{14mu} {and}\mspace{14mu} \frac{\partial\varphi_{\beta}}{\partial p_{a}}} = {\delta\beta}_{\alpha}}$

we can supplement these equations with

$\begin{matrix}{{{\overset{.}{q}}^{(\alpha)} = {\frac{\partial H_{o}}{\partial p_{\alpha}} - {\sum\limits_{\beta = 1}^{N - R_{W}}\; {\frac{\partial g_{\beta}}{\partial p_{\alpha}}{\overset{.}{q}}^{(\beta)}}}}},{\alpha = 1},\ldots \mspace{11mu},{N - R_{W}}} & \left( {E\text{-}23} \right)\end{matrix}$

So we can write

$\begin{matrix}{{{{\overset{.}{q}}^{(i)} = {\frac{\partial H_{o}}{\partial p_{i}} + {\sum\limits_{\beta = 1}^{N - R_{W}}\; {\frac{\partial g_{\beta}}{\partial p_{i}}{\overset{.}{q}}^{(\beta)}}}}},{i = 1},\ldots \mspace{11mu},N}{{{\overset{.}{p}}_{i} = {{- \frac{\partial H_{o}}{\partial q^{(i)}}} - {\sum\limits_{\beta = 1}^{N - R_{W}}\; {{\overset{.}{q}}^{(\beta)}\frac{\partial g_{\beta}}{\partial q^{(i)}}}}}},{i = 1},\ldots \mspace{11mu},N}} & \left( {E\text{-}24} \right)\end{matrix}$

For consistency with (E--11) we should write

$\begin{matrix}{{{\overset{.}{q}}^{(\alpha)} = {\frac{}{t} - {g_{\alpha}\left( {q,\left\{ p_{a} \right\}} \right)}}},{\alpha = 1},\ldots \mspace{11mu},{N - R_{W}}} & \left( {E\text{-}25} \right)\end{matrix}$

where {dot over (p)}_(α) is given by the right hand side of (E--22).

Streamlining the Hamiltonian Equation of Motion (EOM)

Definition 3.16-1. A function f is weakly equal to g denoted by f≈g, iff and g are equal on the subspace defined by the primary constraints,

φ_(β)=0 when f|_(Γ) _(p) =g| _(Γ) _(p)

and

f(q,p)≈g(q,p)

f(q,p)=g(q,p) when {φ_(α)(q,p)=0}

Theorem 3.16.3. Assume f, g are defined over the entire space spanned by{q^((i))}, {p_(i)}. Then if

$\begin{matrix}{{{{f\left( {q,p} \right)}_{\;_{\Gamma_{p}}}} = {{g\left( {q,p} \right)}_{\Gamma_{p}}{Then}}}{{\frac{\partial}{\partial q^{(i)}}\left( {f - {\sum\limits_{\beta}\; {\varphi_{\beta}\frac{\partial f}{\partial p_{\beta}}}}} \right)} \simeq {\frac{\partial}{\partial q^{(i)}}\left( {h - {\sum\limits_{\; \beta}^{\;}\; {\varphi_{\beta}\frac{\partial h}{\partial p_{\beta}}}}} \right)}}{and}} & \left( {E\text{-}26} \right) \\{{\frac{\partial}{\partial p_{i}}\left( {f - {\sum\limits_{\beta}^{\;}\; {\varphi_{\beta}\frac{\partial f}{\partial p_{\beta}}}}} \right)} \simeq {\frac{\partial}{\partial p}\left( {h - {\sum\limits_{\beta}\; {\varphi_{\beta}\frac{\partial h}{\partial p_{\beta}}}}} \right)}} & \left( {E\text{-}27} \right)\end{matrix}$

for i=1, . . . N.

Proof: Consider the two functions f(q, {p_(a)}, {p_(β)}) and h(q,{p_(a)}, {p_(β)}). Using (E--11) and from the hypothesis of the theorem,

f(q,{p _(a) },{g _(α)})=h(q,{p _(a) },{g _(α)})  (E--28)

Thus is follows

$\begin{matrix}{{\left( {\frac{\partial f}{\partial q^{(i)}} + {\sum\limits_{a}^{\;}\; {\frac{\partial f}{\partial p_{a}}\frac{\partial p_{a}}{\partial q^{(i)}}}} + {\sum\limits_{\beta}^{\;}\; {\frac{\partial f}{\partial p_{\beta}}\frac{\partial g_{\beta}}{\partial q^{(i)}}}}} \right)_{\Gamma_{p}} = \left( {\frac{\partial h}{\partial q^{(i)}} + {\sum\limits_{a}^{\;}\; {\frac{\partial h}{\partial p_{a}}\frac{\partial p_{a}}{\partial q^{(i)}}}} + {\sum\limits_{\beta}^{\;}\; {\frac{\partial h}{\partial p_{\beta}}\frac{\partial g_{\beta}}{\partial q^{(i)}}}}} \right)_{\Gamma_{p}}}{and}} & \left( {E\text{-}29} \right) \\{\left( {\frac{\partial f}{\partial p_{i}} + {\sum\limits_{a \neq i}^{\;}\; {\frac{\partial f}{\partial p_{a}}\frac{\partial p_{a}}{\partial p_{i}}}} + {\sum\limits_{\beta}^{\;}\; {\frac{\partial f}{\partial p_{\beta}}\frac{\partial g_{\beta}}{\partial p_{i}}}}} \right)_{\Gamma_{\beta}} = \left( {\frac{\partial h}{\partial p_{i}} + {\sum\limits_{a \neq i}^{\;}\; {\frac{\partial h}{\partial p_{a}}\frac{\partial p_{a}}{\partial p_{i}}}} + {\sum\limits_{\beta}^{\;}\; {\frac{\partial h}{\partial p_{\beta}}\frac{\partial g_{\beta}}{\partial p_{i}}}}} \right)_{\Gamma_{p}}} & \left( {E\text{-}30} \right)\end{matrix}$

Note since φ_(α)(q,p)=p_(α)−g_(α)(q,{p_(a)}), we have

$\frac{\partial g_{\beta}}{\partial q^{(i)}} = {- \frac{\partial{\varphi_{\beta}\left( {q,p} \right)}}{\partial q^{(i)}}}$and$\frac{\partial g_{\beta}}{\partial p_{i}} = \frac{\partial{\varphi_{\beta}\left( {q,p} \right)}}{\partial p_{i}}$and ∂φ_(α)(q, p) = 0

for α=1, . . . , N−R_(W). We have

$\left( {\frac{\partial f}{\partial q^{(i)}} - {\sum\limits_{\; \beta}^{\;}\; {\frac{\partial f}{\partial p_{\beta}}\frac{\partial\varphi_{\beta}}{\partial q^{(i)}}}}} \right)_{\Gamma_{p}} = \left( {\frac{\partial h}{\partial q^{(i)}} - {\sum\limits_{\beta}^{\;}\; {\frac{\partial h}{\partial p_{\beta}}\frac{\partial\varphi_{\beta}}{\partial q^{(i)}}}}} \right)_{\Gamma_{\beta}}$

which can be written as

${\frac{\partial}{\partial q^{(i)}}\left( {f - {\sum\limits_{\beta}\; {\varphi_{\beta}\frac{\partial f}{\partial p_{\beta}}}}} \right)} \simeq {\frac{\partial}{\partial q^{(i)}}\left( {h - {\sum\limits_{\beta}\; {\varphi_{\beta}\frac{\partial h}{\partial p_{\beta}}}}} \right)}$

since φ_(β) ^(∂) ² f/∂p_(β) ²=0 because φ_(β)=0. Similarly,

${\frac{\partial}{\partial p_{i}}\left( {f - {\sum\limits_{\beta}\; {\varphi_{\beta}\frac{\partial f}{\partial p_{\beta}}}}} \right)} \simeq {\frac{\partial}{\partial p_{i}}\left( {h - {\sum\limits_{\beta}\; {\varphi_{\beta}\frac{\partial h}{\partial p_{\beta}}}}} \right)}$

Corrolary 3.16-1.

${\overset{.}{q}}^{(i)} = {\frac{\partial H}{\partial p_{i}} + {\sum\limits_{\beta}{v^{(\beta)}\frac{\partial\varphi_{\beta}}{\partial p_{i}}}}}$${\overset{.}{p}}_{i} = {{- \frac{\partial H}{\partial q^{(i)}}} - {\sum\limits_{\beta}{v^{(\beta)}\frac{\partial\varphi_{\beta}}{\partial q^{(i)}}}}}$

for i=1, . . . , N.

Proof. We consider two Hamiltonians H({q^((i))},{p_(i)}) andH_(o)({q^((i))},{p_(a)}). Define H({q^((i))},{p_(i)}) as follows

H({(q ^((i)) },{p _(i)})≈H _(o)({q ^((i)) },{p _(a)}).

Then using the result of Theorem 3.16.1, from (E--29) with f=H andh=H_(o)

$\begin{matrix}{\frac{\partial H_{o}}{\partial q^{(i)}} \approx {\frac{\partial}{\partial q^{(i)}}\left( {H - {\sum\limits_{\beta = 1}^{N - R_{w}}\; {\varphi_{\beta}\frac{\partial H}{\partial p_{\beta}}}}} \right)}} & \left( {E\text{-}31} \right) \\{\frac{\partial H_{o}}{\partial p_{i}} \approx {\frac{\partial}{\partial p_{i}}\left( {H - {\sum\limits_{\beta = 1}^{N - R_{w}}\; {\varphi_{\beta}\frac{\partial H}{\partial p_{\beta}}}}} \right)}} & \left( {E\text{-}32} \right)\end{matrix}$

Using (E--31) and (E--32) in (E--24), we get

$\begin{matrix}{{\overset{.}{q}}^{(i)} \approx {{\frac{\partial}{\partial p_{i}}\left( {H - {\sum\limits_{\beta}\; {\varphi_{\beta}\frac{\partial H}{\partial p_{\beta}}}}} \right)} + {\sum\limits_{\beta}{{\overset{.}{q}}^{(\beta)}\frac{\partial\varphi_{\beta}}{\partial p_{i}}}}}} & \left( {E\text{-}33} \right) \\{and} & \; \\{{\overset{.}{p}}_{i} \approx {{{- \frac{\partial}{\partial q^{(i)}}}\left( {H - {\sum\limits_{\beta}\; {\varphi_{\beta}\frac{\partial H}{\partial p_{\beta}}}}} \right)} - {\sum\limits_{\beta}{{\overset{.}{q}}^{(\beta)}\frac{\partial\varphi_{\beta}}{\partial q^{(i)}}}}}} & \; \\{or} & \; \\{{\overset{.}{q}}^{(i)} \approx {\frac{\partial}{\partial p_{i}}\left( {H - {\sum\limits_{\beta}\; {\varphi_{\beta}\left( {\frac{\partial H}{\partial p_{\beta}} - {\overset{.}{q}}^{(\beta)}} \right)}}} \right)}} & \; \\{and} & \; \\{{\overset{.}{p}}_{i} \approx {{- \frac{\partial}{\partial q^{(i)}}}\left( {H - {\sum\limits_{\beta}\; {\varphi_{\beta}\left( {\frac{\partial H}{\partial p_{\beta}} - {\overset{.}{q}}^{(\beta)}} \right)}}} \right)}} & \; \\{Define} & \; \\{{v_{\beta} \equiv {{\overset{.}{q}}^{(\beta)} - \frac{\partial H}{\partial p_{\beta}}}}{H_{T} \equiv {H + {\sum\limits_{\beta}\; {v^{(\beta)}\varphi_{\beta}}}}}} & \;\end{matrix}$

So (E--33) becomes

$\begin{matrix}{{{\overset{.}{q}}^{(i)} \approx \frac{\partial H_{T}}{\partial p_{i}}}{{\overset{.}{p}}_{i} \approx {- \frac{\partial H_{T}}{\partial q^{(i)}}}}} & \left( {E\text{-}34} \right)\end{matrix}$

Constrained Hamiltonian Systems

Local symmetries on a Lagrangian based model. Consider

q ^((i)) →q ^((i))(t)+δq ^((i))(t)

{dot over (q)} ^((i)) →{dot over (q)} ^((i))(t)+δ{dot over (q)}^((i))(t)

with i=1, . . . , N. The action of the system is given by

S(q,{dot over (q)})=∫L(q,{dot over (q)})dt

where q and {dot over (q)} are n-dimensional column vectors. The actiondifferential

$\begin{matrix}{{\delta \; S} = {{\int{{L\left( {{q + {\delta \; q}},{\overset{.}{q} + {\delta \; \overset{.}{q}}}} \right)}{t}}} - {\int{{L\left( {q,\overset{.}{q}} \right)}{t}}}}} \\{= {{\int{{L\left( {{q + {\delta \; q}},{\overset{.}{q} + {\delta \; \overset{.}{q}}}} \right)}{t}}} - {\int{{L\left( {q,\overset{.}{q}} \right)}{t}}}}} \\{= {\int{\left\lbrack {{\sum\limits_{i}\; {\frac{\partial L}{\partial q^{(i)}}\delta \; q^{(i)}}} + {\sum\limits_{i}\; {\frac{\partial L}{\partial{\overset{.}{q}}^{(i)}}\delta \; {\overset{.}{q}}^{(i)}}}} \right\rbrack {t}}}} \\{= {- {\int{\sum\limits_{i}{\left\lbrack {{\frac{}{t}\frac{\partial L}{\partial{\overset{.}{q}}^{(i)}}} - \frac{\partial L}{\partial q^{(i)}}} \right\rbrack \delta \; q^{(i)}{t}}}}}} \\{= {- {\sum\limits_{\;}\; {{t}{\sum\limits_{i}{{E_{i}^{(o)}\left( {q,\overset{.}{q},\overset{¨}{q}} \right)}\delta \; q^{(i)}}}}}}}\end{matrix}\quad$

where we define the Euler-Lagrange differential operator

${E_{i}^{(o)}\left( {q,\overset{.}{q},\overset{¨}{q}} \right)} = {{\frac{}{t}\frac{\partial L}{\partial{\overset{.}{q}}^{(i)}}} - {\frac{\partial L}{\partial q^{(i)}}.}}$

Note that

$\begin{matrix}{{\int{\sum\limits_{i = 1}^{N}\; {{E_{i}^{(o)}\left( {q,\overset{.}{q},\overset{¨}{q}} \right)}\delta \; q^{(i)}{t}}}} \equiv 0} & \left( {3.17\text{-}1} \right)\end{matrix}$

on shell. Expanding E_(i) ^((o))

$\begin{matrix}{{E_{i}^{(o)}\left( {q,\overset{.}{q},\overset{¨}{q}} \right)} = {{\sum\limits_{j}\left\lbrack {{\frac{\partial^{2}{L\left( {q,\overset{.}{q}} \right)}}{{\partial{\overset{.}{q}}^{(i)}}{\partial{\overset{¨}{q}}^{(j)}}}{\overset{¨}{q}}^{(i)}} + {\frac{\partial^{2}{L\left( {q,\overset{.}{q}} \right)}}{{\partial{\overset{.}{q}}^{(i)}}{\partial q^{(j)}}}{\overset{.}{q}}^{(i)}}} \right\rbrack} - \frac{\partial{L\left( {q,\overset{.}{q}} \right)}}{\partial q^{(i)}}}} \\{= {{\sum\limits_{j}{{W_{ij}\left( {q,\overset{.}{q}} \right)}{\overset{¨}{q}}^{(j)}}} + {\sum\limits_{j}{\frac{\partial^{2}{L\left( {q,\overset{.}{q}} \right)}}{{\partial{\overset{.}{q}}^{(i)}}{\partial q^{(j)}}}{\overset{.}{q}}^{(i)}}} - \frac{\partial{L\left( {q,\overset{.}{q}} \right)}}{\partial q^{(i)}}}} \\{= {{\sum\limits_{j}{{W_{ij}\left( {q,\overset{.}{q}} \right)}{\overset{¨}{q}}^{(j)}}} + {k_{i}\left( {q,\overset{.}{q}} \right)}}}\end{matrix}$

If L is singular, W_((N×N)) is not invertible so (3.17-1) cannot besolved for {umlaut over (q)}_(i), =1, . . . , N. If Rank(W(q,{dot over(q)}))=R_(W) on shell, then there exist N−R_(W) in the theory. Thereexist N−R_(W) independent left (or right) zero mode eigenvectors w_(i)^((o,k)), i=1, . . . , N−R_(W) such that

$\begin{matrix}{{{{\sum\limits_{i}{{w_{i}^{({o,k})}\left( {q,\overset{.}{q}} \right)}{W_{ij}\left( {q,\overset{.}{q}} \right)}}} = 0},{k = 1},\ldots \mspace{11mu},{N - R_{W}}}{Thus}{\varphi^{({o,k})} = {\sum\limits_{i = 1}^{N}\; {{w_{i}^{({o,k})}\left( {q,\overset{.}{q}} \right)}{E_{i}^{(o)}\left( {q,\overset{.}{q},\overset{¨}{q}} \right)}}}}} & \left( {3.17\text{-}2} \right)\end{matrix}$

depend on q and {dot over (q)} only. The φ^((o,k)) also vanish on shell:

φ^((o,k))(q,{dot over (q)})=0, k=1, . . . ,N−R _(W)

The set {φ^((o,k))|k=1, . . . , N−R_(W)} are the zero generationconstraints. It is possible that not all the {φ^((o,k))} are linearlyindependent. So we may find linear combinations of the zero modeeigenvectors

$v_{i}^{({o,n_{o}})} = {\sum\limits_{k}\; {c_{k}^{(n_{o})}w_{i}^{({o,k})}}}$

such that we have

G ^((o,n) ^(o) ⁾ =v ^((o,n) ^(o) ⁾ E ^((o))≡0, n _(o) , . . . ,N_(o)  (3.17-3)

These are called gauge identities.

Any variation δq_(i), i=1, . . . , N, of the form

${\delta \; q_{i}} = {\sum\limits_{n_{o}}\; {ɛ_{n_{o}}v_{i}^{({o,n_{o}})}}}$

Is action invariant by (3.17-1).

Given this definition of δq_(i) and (3.17-3), we conclude

$\begin{matrix}{{\delta \; S} = {\int{{t}{\sum\limits_{i = 1}^{N}\; {{E_{i}^{(o)}\left( {q,\overset{.}{q},\overset{¨}{q}} \right)}{\sum\limits_{n_{o}}\; {{ɛ_{n_{o}}(t)}\; v_{i}^{({o,n_{o}})}}}}}}}} \\{= {\int{{t}{\sum\limits_{i = 1}^{N}{ɛ_{n_{o}}{\sum\limits_{n_{o}}{{E_{i}^{(o)}\left( {q,\overset{.}{q},\overset{¨}{q}} \right)}{v_{i}^{({o,n_{o}})}\left( {q,\overset{.}{q}} \right)}}}}}}}} \\{= {\int{{t}{\sum\limits_{i = 1}^{N}{ɛ_{n_{o}}G^{({o,n_{o}})}}}}}} \\{\equiv 0}\end{matrix}\quad$

everywhere. The remaining zero generating modes which we denote byu^((o,n) ⁰ ⁾ lead to genuine constraints. They are of the form φ^((o,n)^(o) ⁾(q,{dot over (q)})=0 on shell, where

φ^((o,n) ^(o) ⁾ =u ^((o,n) ^(o) ⁾ E ^((o)).  (3.17-4)

The algorithm now proceeds as follows. We separate the gauge identities(3.17-3) from the nontrivial constraints (3.17-4) and will list themseparately. They will be used for determining local symmetrytransformations.

Next we want to search for additional constraints. We do this bysearching for further functions of the coordinates and velocities whichvanish in the space of physical trajectories. To this effect considerthe following N+N_(o) vector constructed from E^((o)) and the timederivative of the constraints (3.17-4)

$\begin{matrix}{\left\lbrack E^{(1)} \right\rbrack = {\begin{bmatrix}E^{(o)} \\{\frac{}{t}\left( {u^{({o,1})}E^{(o)}} \right)} \\\vdots \\{\frac{}{t}\left( {u^{({o,n_{o}})}E^{(o)}} \right)}\end{bmatrix} = \begin{bmatrix}E^{(o)} \\{\frac{}{t}\varphi^{(o)}}\end{bmatrix}}} & \left( {3.17\text{-}5} \right)\end{matrix}$

by construction. The constraint φ^((o)) is valid for all time andtherefore

${\frac{}{t}\varphi^{(o)}} = 0$

on shell, but

$\begin{matrix}{{\frac{\varphi^{({o,i})}}{t} = {{{\nabla{\overset{.}{q}\left( {u^{({o,i})}E^{(o)}} \right)}}\overset{¨}{q}} + {{\nabla{q\left( {u^{({o,i})}E^{(o)}} \right)}}\overset{.}{q}}}}{{{So}\left\lbrack E_{i_{1}}^{(1)} \right\rbrack} = {{\sum\limits_{j = 1}^{n}{{W_{i_{1}j}^{(1)}\left( {q,\overset{.}{q}} \right)}{\overset{.}{q}}^{(j)}}} + {k_{i_{1}}^{(1)}\left( {q,\overset{.}{q}} \right)}}}} & \left( {3.17\text{-}6} \right)\end{matrix}$

where i₁=1, . . . , N+N_(o), and

$\begin{matrix}{\left\lbrack W_{i_{1}i}^{(1)} \right\rbrack = {{\begin{bmatrix}W^{(o)} \\{\nabla{\overset{.}{q}\left( {u^{({o,i})}E^{(o)}} \right)}} \\{\nabla{\overset{.}{q}\left( {u^{({o,N_{o}})}E^{(o)}} \right)}}\end{bmatrix}\left\lbrack k_{i_{1}}^{(1)} \right\rbrack} = \begin{bmatrix}k^{(o)} \\{\sum\limits_{i}{\frac{\partial}{\partial q^{(j)}}\left( {u^{({o,i})}E^{(o)}} \right){\overset{.}{q}}^{(j)}}} \\\vdots \\{\sum\limits_{i}{\frac{\partial}{\partial q^{(j)}}\left( {u^{({o,N_{o}})}E^{(o)}} \right){\overset{.}{q}}^{(j)}}}\end{bmatrix}}} & \left( {3.17\text{-}7} \right)\end{matrix}$

We next look for the zero modes of W⁽¹⁾. By construction, these zeromodes include the o modes of the previous level. The gauge identities atlevel 1 are.

$\begin{matrix}{G^{({1,n_{1}})} = {{{v^{({1,n_{1}})}E^{1}} - {\sum\limits_{n_{o} = 1}^{N_{o}}{M_{n_{1}n_{o}}^{({1,0})}\left( {u^{({o,n_{o}})}E^{(o)}} \right)}}} \equiv 0}} & \left( {3.17\text{-}8} \right)\end{matrix}$

where n₁=1, . . . , N₁ and the genuine constraints are of the form

φ^((1,n) ¹ ⁾=φ^((1,n) ¹ ⁾ E ¹=0  (3.17-9)

with n₁=1, . . . , N₁ on shell.

We next adjoin the new identities (3.17-8) to the ones determinedearlier (3.17-3) with the remaining constraints (3.17-9) we proceed asbefore, adjoining their time derivatives to (3.17-5) and construct W_(i)₁ _(i) ⁽¹⁾ and k_(i) ₁ ⁽¹⁾.

The iterative process will terminate at some level M if either i) thereis not further zero modes, or ii) the new constraints can be expressedas linear combinations of previous constraints.

The Maximal Set of Linearly Independent Gauge Identities Generated bythe Algorithm

Note that the algorithm steps are of the form

$\begin{matrix}{G^{({o,n_{o}})} = {{u^{({o,n_{o}})}E^{(o)}} \equiv 0}} & \left( {3.17\text{-}10} \right) \\{G^{({l,n_{l}})} = {{u^{({l,n_{l}})}E^{(l)}} - {\sum\limits_{l^{\prime} = 0}^{l - 1}{\sum\limits_{n_{l^{\prime}} = 0}^{N_{l^{\prime}}}{M_{n_{l}n_{l^{\prime}}}^{({l,l^{\prime}})}\varphi^{({l^{\prime},n_{l^{\prime}}})}}}}}} & \left( {3.17\text{-}11} \right)\end{matrix}$

with L=1, . . . , N_(l). The M_(n) _(l) _(n) _(l′) ^((l,l′)), are onlyfunctions of q and {dot over (q)}. And

$\begin{matrix}{{\varphi^{({l,n_{l}})} = {u^{({l,n_{l}})}E^{(l)}}},{n_{1} = 1},\ldots \mspace{14mu},N_{l},} & \left( {3.17\text{-}12} \right) \\{E^{(l)} = \begin{bmatrix}E^{(o)} \\\frac{\varphi^{(o)}}{t} \\\vdots \\\frac{\varphi^{({l - 1})}}{t}\end{bmatrix}} & \left( {3.17\text{-}13} \right)\end{matrix}$

where φ^((l)) is a column vector with N₁ components φ^((l,n) ^(l) ⁾.Thus we conclude from (3.17-13) and (3.17-11) that the general form ofthe gauge identity given by (3.17-11) is of the form

$\begin{matrix}{G^{({l,n_{l}})} = {{\sum\limits_{i = 1}^{N_{l}}{\sum\limits_{l = 1}^{M}{\sum\limits_{m = 1}^{l}{\varsigma_{mi}^{({l,m_{l}})}\frac{^{m}}{t^{m}}E_{i}^{(o)}}}}} \equiv 0}} & \left( {3.17\text{-}14} \right)\end{matrix}$

where δ_(mi) ^((l,m) ¹ ⁾(q,{dot over (q)}) and N_(l)<M. From (3.17-14)it also follows that

$\begin{matrix}{{\sum\limits_{l = 1}^{M}{\sum\limits_{n_{l} = 1}^{l}{ɛ^{({l,n_{l}})}G^{({l,n_{l}})}}}} \equiv 0} & \left( {3.17\text{-}15} \right)\end{matrix}$

This identity can also be written as

$\begin{matrix}{{{\sum{\delta \; q^{(i)}E_{i}^{(o)}}} - {\frac{}{t}F}}{where}{{\delta \; q^{(i)}} = {\sum\limits_{l = 1}^{M}{\sum\limits_{n_{l} = 1}^{N_{l}}{\sum\limits_{m = q}^{l}{\left( {- 1} \right)^{m}\frac{^{m}}{t^{m}}\varsigma_{m}^{({l,n_{l}})}{ɛ^{({l,m_{l}})}(t)}}}}}}} & \left( {3.17\text{-}16} \right)\end{matrix}$

and F is a complicated function of q and {dot over (q)}. By collectingindices l, n_(l) together

${\delta \; q_{i}} = {\sum\limits_{l = 1}^{M}{\sum\limits_{n_{l} = 1}^{N_{l}}{\sum\limits_{m = q}^{l}{\left( {- 1} \right)^{m}\varsigma_{m_{i}}^{(a)}{ɛ^{(a)}(t)}}}}}$

Example of Constrained Hamiltonian System in Lagrangian Form

Let

$\begin{matrix}{{L\left( {q,\overset{.}{q}} \right)} = {{\frac{1}{2}{\overset{.}{q}}^{2{(1)}}} + {{\overset{.}{q}}^{(1)}q^{(2)}} + {\frac{1}{2}\left( {q^{(1)} - q^{(2)}} \right)^{2}}}} & \left( {3.17\text{-}17} \right) \\{E^{(o)} = {\begin{bmatrix}{{\frac{}{t}\frac{\partial}{\partial{\overset{.}{q}}^{(1)}}} - \frac{\partial L}{\partial q^{(1)}}} \\{{\frac{}{t}\frac{\partial}{\partial{\overset{.}{q}}^{(2)}}} - \frac{\partial L}{\partial q^{(2)}}}\end{bmatrix} = \begin{bmatrix}{{\overset{¨}{q}}^{(1)} + {2q^{(2)}} - q^{(1)}} \\{q^{(1)} - q^{(2)}}\end{bmatrix}}} & \left( {3.17\text{-}18} \right) \\{W = \begin{bmatrix}1 & 0 \\0 & 0\end{bmatrix}} & \left( {3.17\text{-}19} \right) \\{k = \begin{bmatrix}{{\overset{.}{q}}^{(2)} - q^{(1)} + q^{(2)}} \\{{- {\overset{.}{q}}^{(1)}} - q^{(2)} + q^{(1)}}\end{bmatrix}} & \left( {3.17\text{-}20} \right)\end{matrix}$

The only o mode is

$\begin{matrix}{{u^{(o)} = \left\lbrack {0,1} \right\rbrack}{Then}\begin{matrix}{E^{(o)} = {{W^{(o)}\overset{¨}{q}} + k^{(o)}}} \\{= {{\begin{bmatrix}1 & 0 \\0 & 0\end{bmatrix}\begin{bmatrix}{\overset{¨}{q}}^{(1)} \\{\overset{¨}{q}}^{(2)}\end{bmatrix}} + \begin{bmatrix}{{\overset{.}{q}}^{(2)} - q^{(1)} + q^{(2)}} \\{{- {\overset{.}{q}}^{(1)}} - q^{(2)} + q^{(1)}}\end{bmatrix}}}\end{matrix}{Then}\begin{matrix}{{u^{(o)}E^{(o)}} = {\begin{bmatrix}0 & 1\end{bmatrix}\left\lbrack {{\begin{bmatrix}1 & 0 \\0 & 0\end{bmatrix}\begin{bmatrix}{\overset{¨}{q}}^{(1)} \\{\overset{¨}{q}}^{(2)}\end{bmatrix}} + \begin{bmatrix}{{\overset{.}{q}}^{(2)} - q^{(1)} + q^{(2)}} \\{{- {\overset{.}{q}}^{(1)}} - q^{(2)} + q^{(1)}}\end{bmatrix}} \right\rbrack}} \\{= {{- {\overset{.}{q}}^{(1)}} - q^{(2)} + q^{(1)}}} \\{= 0}\end{matrix}} & \left( {3.17\text{-}21} \right)\end{matrix}$

on shell. Then there are no gauge identities for E^((o)). Now constructE⁽¹⁾.

$E^{(1)} = {\begin{bmatrix}E^{(o)} \\{\frac{t}{t}u^{(o)}E^{(o)}}\end{bmatrix} = \begin{bmatrix}{{\overset{.}{q}}^{(2)} - q^{(1)} + q^{(2)}} \\{{- {\overset{.}{q}}^{(1)}} - q^{(2)} + q^{(1)}} \\{{- {\overset{¨}{q}}^{(1)}} - {\overset{.}{q}}^{(2)} + {\overset{.}{q}}^{(1)}}\end{bmatrix}}$

which can be written

$\begin{matrix}{E^{(1)} = {{W^{(1)}\overset{¨}{q}} + k^{(1)}}} \\{= {{\begin{bmatrix}0 & 0 \\0 & 0 \\{- 1} & 0\end{bmatrix}\begin{bmatrix}{\overset{¨}{q}}^{(1)} \\{\overset{¨}{q}}^{(2)}\end{bmatrix}} + \begin{bmatrix}{{\overset{.}{q}}^{(2)} - q^{(1)} + q^{(2)}} \\{{- {\overset{.}{q}}^{(1)}} - q^{(2)} + q^{(1)}} \\{{- {\overset{¨}{q}}^{(2)}} + {\overset{.}{q}}^{(1)}}\end{bmatrix}}}\end{matrix}$

There zero modes of w⁽¹⁾ are

$W^{(1)}\left\{ \begin{matrix}\begin{bmatrix}0 & 1 & 0\end{bmatrix} \\\begin{bmatrix}1 & 0 & 1\end{bmatrix}\end{matrix} \right.$

The first zero mode is the previous one augmented by one dimension andreproduces the previous constraint. The second mode reproduces thenegative of the constraint (3.17-21). That is,

v ⁽¹⁾ E ⁽¹⁾ =−u ^((o)) E ^((o))

with v⁽¹⁾=[1 0 1]. This leads to the gauge identity

G ⁽¹⁾ =v ⁽¹⁾ E ⁽¹⁾ +u ^((o)) E ^((o))≡0

Companionship: Reconciling Agents in the Network.

The outline of the companionship process is as follows for a system of Nagents.

-   -   Determine the state action space of the system for N−1 agents to        create a Tellegen decision element.    -   Update the remaining agent with the Tellegen DE.    -   Repeat process so that all N agents are updated with respect to        their Tellegen DEs.    -   User submits query.    -   System used KB to establish equations of motion for system in        Lagrangian or Hamiltonian form.    -   System determines optimal trajectory via optimization algorithm        of the equations of motion that conform to the principle of        least action.    -   System returns solution which is a point in the phase space and        also serves as an answer to the query.

It will also be appreciated that in some embodiments the functionalityprovided by the routines discussed above may be provided in alternativeways, such as being split among more routines or consolidated into fewerroutines. Similarly, in some embodiments illustrated routines mayprovide more or less functionality than is described, such as when otherillustrated routines instead lack or include such functionalityrespectively, or when the amount of functionality that is provided isaltered. In addition, while various operations may be illustrated asbeing performed in a particular manner (e.g., in serial or in parallel,synchronously or asynchronously, etc.) and/or in a particular order,those skilled in the art will appreciate that in other embodiments theoperations may be performed in other orders and in other manners. Thoseskilled in the art will also appreciate that the data structuresdiscussed above may be structured in different manners, such as byhaving a single data structure split into multiple data structures or byhaving multiple data structures consolidated into a single datastructure. Similarly, in some embodiments illustrated data structuresmay store more or less information than is described, such as when otherillustrated data structures instead lack or include such informationrespectively, or when the amount or types of information that is storedis altered.

From the foregoing it will be appreciated that, although specificembodiments have been described herein for purposes of illustration,various modifications may be made without deviating from the spirit andscope of the invention. Accordingly, the invention is not limited exceptas by the appended claims and the elements recited therein. In addition,while certain aspects of the invention are presented below in certainclaim forms, the inventors contemplate the various aspects of theinvention in any available claim form. For example, while only someaspects of the invention may currently be recited as being embodied in acomputer-readable medium, other aspects may likewise be so embodied.

What is claimed is:
 1. A computer-implemented method comprising:obtaining, by one or more computing systems of a collaborativedistributed decision system, coupled differential equations thatrepresent a current state of a physical system and that are generatedfrom system information and objective information and sensorinformation, wherein the physical system has a plurality ofinter-related elements and has one or more outputs whose values varybased at least in part on values of one or more manipulatable controlelements of the plurality, wherein the system information is supplied byone or more users to describe the physical system and includes multiplerules that each has one or more conditions to evaluate and that specifyrestrictions involving the plurality of elements, wherein the objectiveinformation identifies a goal to be achieved during controlling of thephysical system, and wherein the sensor information identifies currentstate information for at least one element of the plurality and partialinitial state information for the physical system at an earlier time;performing, by the one or more computing systems, a piecewise linearanalysis of the coupled differential equations to identify one or morecontrol actions to take in the physical system that manipulate values ofthe one or more manipulatable control elements and that provide asolution for the goal within a threshold amount of an optimal solutionfor the goal, wherein the performing of the piecewise linear analysisincludes: dividing a time window from the earlier time to the specifiedtime into a succession of a plurality of time slices that each, otherthan a first time slice of the succession, overlaps at least in partwith a prior time slice of the succession; evaluating, based on thepartial initial state information, the coupled differential equations toidentify an initial solution for the goal for the first time slice thatincludes simulating effects of manipulating the one or moremanipulatable control elements to one or more initial values, andstoring a model describing a state of the physical system for the firsttime slice that includes the simulated effects of the manipulating tothe one or more initial values; for each time slice of the successionafter the first time slice, updating the stored model for the prior timeslice to reflect an additional solution for the goal for the time slicethat includes simulating effects of further manipulating the one or moremanipulatable control elements to one or more additional values; andafter updating the stored model to reflect the additional solution forthe goal for a last time slice of the succession, further updating thestored model for the last time slice to reflect a further solution forthe goal for a next time period after the time window based at least inpart on the identified current state information, wherein the furthersolution includes the identified one or more control actions to take inthe physical system for a current time; and providing information aboutthe identified one or more control actions, to enable actions to betaken in the physical system for the current time to affect the outputsbased on the identified one or more control actions.
 2. Thecomputer-implemented method of claim 1 wherein the physical system is anelectricity generating facility, wherein the plurality of inter-relatedelements include multiple alternative electricity sources within theelectricity generating facility, wherein the manipulatable controlelements include one or more controls to determine whether to accept arequest to supply a specified amount of electricity at the current timeand to select which alternative electricity source to provide thespecified amount of electricity at the current time if accepted, whereinthe outputs include the electricity being provided, and wherein the goalincludes to maximize profits for the electricity generating facilityfrom providing of the electricity.
 3. The computer-implemented method ofclaim 1 wherein the physical system is an energy generating facility,wherein the plurality of inter-related elements include at least oneenergy source within the energy generating facility and at least oneenergy storage mechanism within the energy generating facility, whereinthe manipulatable control elements include one or more controls todetermine whether to accept a request to supply a specified amount ofenergy at the current time and to determine to provide energy to the atleast one energy storage mechanism at the current time if not acceptedand to provide energy from the at least one energy source at the currenttime if accepted, wherein the outputs include the energy being provided,and wherein the goal includes to maximize profits for the electricitygenerating facility from providing of the energy.
 4. Thecomputer-implemented method of claim 1 wherein the physical system is avehicle, wherein the plurality of inter-related elements include a motorand a battery of the vehicle, wherein the manipulatable control elementsinclude one or more controls to select whether at the current time toremove energy from the battery to power the motor or to add excessenergy to the battery and how much energy to remove from the battery,wherein the outputs include effects of the motor to move the vehicle atthe current time, and wherein the goal includes to move the vehicle atone or more specified speeds with a minimum of energy produced from thebattery.
 5. The computer-implemented method of claim 4 wherein theplurality of inter-related elements further includes an engine that ismanipulatable to modify energy generated from the engine, wherein themanipulatable control elements further include one or more additionalcontrols to determine how much energy to generate from the engine foruse at least in part in adding the excess energy to the battery, andwherein the goal further includes to minimize use of fuel by the engine.6. The computer-implemented method of claim 1 wherein the physicalsystem includes product inventory at one or more locations, wherein theplurality of inter-related elements include one or more product sourcesthat provide products and increase the inventory at the one or morelocations and further include one or more product recipients thatreceive products and decrease the inventory at the one or morelocations, wherein the manipulatable control elements include one ormore first controls to select at the current time one or more firstamounts of one or more products to request from the one or more productsources, and further include one or more second controls to select atthe current time one or more second amounts of at least one product toprovide to the one or more product recipients, wherein the outputsinclude products being provided from the one or more locations to theone or more product recipients, and wherein the goal includes tomaximize profit of an entity operating the one or more locations whilemaintaining the inventory at one or more specified levels.
 7. Thecomputer-implemented method of claim 1 wherein the stored model for eachof the time slices of the succession is expressed with a Hamiltonianfunction specific to the time slice, and wherein each updating of thestored model for a prior time slice to reflect an additional solutionfor the goal includes modifying the Hamiltonian function expressed bythe stored model for the prior time slice.
 8. The computer-implementedmethod of claim 1 wherein the updated stored model for the last timeslice is expressed with a Hamiltonian function, and wherein the furtherupdating of the stored model for the last time slice to reflect thefurther solution for the goal for the next time period includesmodifying the Hamiltonian function based at least in part on theidentified current state information.
 9. The computer-implemented methodof claim 1 wherein the evaluation of the coupled differential equationsto identify the initial solution for the goal for the first time sliceand the updating for each time slice of the succession after the firsttime slice of the stored model is performed to train the updated storedmodel for the last time slice to reflect values of at least some of theplurality of inter-related elements for the current time that includeone or more elements whose values are not directly observable, andwherein training of the updated stored model for the last time sliceenables the further updating to reflect the further solution for thegoal for the next time period.
 10. The computer-implemented method ofclaim 1 further comprising, for each of multiple additional times afterthe current time, adapting a current copy of the stored model to reflectthe additional time by: obtaining additional sensor information thatidentifies state information at the additional time for one or moreelements of the plurality; determining, by the one or more computingsystems, if an updated copy of the stored model can be generated for theadditional time by attempting to identify another solution for the goalfor the additional time based at least in part on the additional sensorinformation, wherein the another solution, if identified, includes oneor more further control actions to take in the physical system for theadditional time; and if the updated copy of the stored model can begenerated for the additional time, generating and storing the updatedcopy of the stored model, and providing information about one or moreadditional control actions to perform in the physical system for theadditional time to further manipulate the manipulatable control elementsin a specified manner.
 11. The computer-implemented method of claim 10further comprising, for each of the multiple additional times and if theupdated copy of the stored model for the additional time is notgenerated, providing information about one or more further associatedcontrol actions to perform in the physical system for that additionaltime to further manipulate the manipulatable control elements in aspecified manner, wherein the one or more further associated controlactions are based on the current copy of the stored model beforeupdating for the additional time.
 12. The computer-implemented method ofclaim 11 wherein generating of the updated copy of the stored model forone of the multiple additional times includes a deadline for thegenerating to enable real-time control of the physical system to beperformed based on performing control actions in the physical system forthe one additional time, and wherein the generating of the updated copyof the stored model for the one additional time fails to complete beforethe deadline, such that the one or more further associated controlactions for that one additional time are performed in the physicalsystem for that one additional time to further manipulate themanipulatable control elements in a specified manner.
 13. Thecomputer-implemented method of claim 10 wherein the updated copy of thestored model for one of the multiple additional times is not generatedin a first attempt, and wherein the method further comprises generatingthe updated copy of the stored model for the one additional time duringa second attempt by: determining, by the one or more computing systems,at least one of the multiple rules to temporarily relax by modifying atleast one of the specified restrictions corresponding to the determinedat least one rule; generating, by the one or more computing systems,additional coupled differential equations that represent a current stateof the physical system for the one additional time based at least inpart on the modified at least one specified restrictions for thedetermined at least one rule; performing, by the one or more computingsystems, a further piecewise linear analysis of the generated additionalcoupled differential equations to identify the another solution for thegoal for the one additional time, generating and storing the updatedcopy of the stored model for the one additional time; and providinginformation about the one or more additional control actions of theupdated copy of the stored model for the one additional time to performin the physical system for the one additional time to further manipulatethe manipulatable control elements in a specified manner.
 14. Thecomputer-implemented method of claim 13 wherein the multiple rulesinclude one or more absolute rules that specify non-modifiablerestrictions that are requirements regarding operation of the physicalsystem, and further include one or more hard rules that specifyrestrictions regarding operation of the physical system that can bemodified in specified situations, and wherein each determined at leastone rule is one of the hard rules.
 15. The computer-implemented methodof claim 13 wherein the multiple rules include one or more soft ruleswhose conditions evaluate to one of three or more possible values underdiffering situations to represent varying degrees of uncertainty andfurther include additional rules whose conditions evaluate to eithertrue or false under differing situations, and wherein one or more of thedetermined at least one rules are from the soft rules.
 16. Thecomputer-implemented method of claim 10 wherein the determining if theupdated copy of the stored model can be generated for one of themultiple additional times includes generating values for one or moremodel error measurements based at least in part on the current copy ofthe stored model for the one additional time, and includes determiningthat at least one of the generated values exceeds an error threshold,and wherein the method further comprises generating the updated copy ofthe stored model for the one additional time by: evaluating, by the oneor more computing systems, the generated values for the one or moremodel error measurements to determine at least one of the multiple rulesthat is incorrect; modifying, by the one or more computing systems, thedetermined at least one rule in a manner expected to reduce values forthe one or more model error measurements; generating, by the one or morecomputing systems, additional coupled differential equations thatrepresent a current state of the physical system for the one additionaltime based at least in part on the modified determined at least onerule; performing, by the one or more computing systems, a furtherpiecewise linear analysis of the generated additional coupleddifferential equations to identify the another solution for the goal forthe one additional time, generating and storing the updated copy of thestored model for the one additional time; and providing informationabout the one or more additional control actions of the updated copy ofthe stored model for the one additional time to perform in the physicalsystem for the one additional time to further manipulate themanipulatable control elements in a specified manner.
 17. Thecomputer-implemented method of claim 16 wherein the another solutionidentified for the goal for two or more of the additional times each hasan associated error measurement within a defined threshold relative toan optimal solution for the goal for that additional time, and whereinthe one or more model error measurements are based on a rate of changeof one or more of: Hamiltonian functions expressed by two or more copiesof the model for two or more times; amounts of entropy included in twoor more copies of the model for two or more times; values of variablesassociated with the plurality of inter-related elements in stateinformation for the physical system for two or more times; or areduction in the associated error measurements for the another solutionsfor the two or more additional times.
 18. The computer-implementedmethod of claim 10 wherein the determining if the updated copy of thestored model can be generated for one of the multiple additional timesincludes: generating, by the one or more computing systems, the updatedcopy of the stored model for the one additional time; generating, by theone or more computing systems, values for one or more model errormeasurements for the generated updated copy of the stored model for theone additional time; determining, by the one or more computing systems,that at least one of the generated values exceeds an error threshold;and replacing, by the one or more computing systems, the generatedupdated copy of the stored model for the one additional time with a newgenerated updated copy of the stored model for the one additional time,by causing a new copy of the model for the one additional time to begenerated without using any past copies of the model, and storing thegenerated new copy of the model for the one additional time.
 19. Thecomputer-implemented method of claim 1 further comprising modifying themultiple rules during one of the multiple additional times, and whereinupdating of copies of the stored model after the one additional timeincludes using the modified rules.
 20. The computer-implemented methodof claim 1 further comprising, before the dividing of the time windowinto the succession of time slices, determining, by the one or morecomputing systems, at least one of a size of the time slices or a sizeof the time window.
 21. The computer-implemented method of claim 20wherein the determining of the at least one of the size of the timeslices or the size of the time window includes evaluating multiple testsizes for the at least one of the size of the time slices or the size ofthe time window, and selecting, based at least in part on theevaluating, one or more of the multiple sizes to use for the determinedat least one size of the time slices or size of the time window.
 22. Thecomputer-implemented method of claim 20 further comprising generating aHamiltonian function to express a copy of the model of the state of thephysical system before the dividing of the time window into thesuccession of time slices, and wherein the determining of the at leastone of the size of the time slices or the size of the time windowincludes performing a symbolic computation analysis of the Hamiltonianfunction to identify one or more preferred sizes to use for thedetermined at least one size of the time slices or size of the timewindow.
 23. The computer-implemented method of claim 1 wherein theproviding of the information about the identified one or more controlactions includes performing, by the one or more computing systems, theactions in the physical system to affect the outputs by manipulating themanipulatable control elements in specified manners for the identifiedone or more control actions.
 24. A non-transitory computer-readablemedium having stored contents that cause one or more computing systemsto perform a method, the method comprising: obtaining, by the one ormore computing systems, coupled differential equations that represent astate of a target system at a specified time and that are generated fromsystem information and objective information and sensor information,wherein the target system has a plurality of elements that areinter-related and that include one or more control elements withmodifiable values, wherein the system information is supplied by one ormore users to describe the physical system and includes restrictionsinvolving the plurality of elements, wherein the objective informationidentifies a goal to be achieved based at least in part on modifying thevalues of the control elements, and wherein the sensor informationidentifies state information for the specified time for at least oneelement of the plurality; performing, by the one or more computingsystems, a piecewise linear analysis of the coupled differentialequations to identify a solution for the goal for the specified timewithin a threshold amount of an optimal solution for the goal, whereinthe identified solution has one or more associated control actions thatmodify at least one value of at least one of the control elements in aspecified manner, and wherein the performing of the piecewise linearanalysis includes: dividing a time window from an earlier time to thespecified time into a succession of a plurality of time slices;evaluating, based on initial state information for the earlier time, thecoupled differential equations to identify an initial solution for thegoal for a first time slice of the succession that includes simulatingeffects of modifying one or more values of the one or more manipulatablecontrol elements in a specified initial manner, and storing a modeldescribing a state of the target system for the first time slice thatincludes the simulated effects of the modifying of the one or morevalues; for each time slice of the succession after the first timeslice, updating the stored model for a prior time slice to reflect anadditional solution for the goal for the time slice that includessimulating effects of further modifying one or more values of the one ormore manipulatable control elements; and after updating the stored modelto reflect the additional solution for the goal for a last time slice ofthe succession, further updating the stored model for the last timeslice to reflect a further solution for the goal for a next time periodafter the time window based at least in part on the identified stateinformation for the specified time, wherein the further solutionincludes the one or more associated control actions; and providinginformation about the one or more associated control actions, to enablemodification of the at least one value of the at least one controlelement for the specified time based on the one or more associatedcontrol actions.
 25. The non-transitory computer-readable medium ofclaim 24 wherein the target system is a physical system having one ormore outputs whose values vary based at least in part on the values ofthe control elements, wherein the one or more computing systems are partof a collaborative distributed decision system, and wherein the storedcontents include software instructions that, when executed, furthercause the one or more computing systems to initiate performance of theone or more associated control actions in the physical system to modifythe at least one value of the at least one control element and to causeresulting changes in the values of the one or more outputs for thespecified time.
 26. The non-transitory computer-readable medium of claim24 wherein the target system includes one or more computing resourcesbeing protected from unauthorized operations, wherein the plurality ofinter-related elements include one or more sources of attempts toperform operations, wherein the control elements include one or morecontrols to determine whether a change in authorization to a specifiedtype of operation is needed and to select one or more actions to take toimplement the change in authorization if so determined, and wherein thegoal includes to minimize unauthorized operations that are performed.27. The non-transitory computer-readable medium of claim 24 wherein thetarget system includes one or more information sources to be analyzed todetermine a risk level from information of the one or more informationsources, wherein the control elements include one or more controls todetermine whether the risk level exceeds a specified threshold and toselect one or more actions to take to mitigate the risk level, andwherein the goal includes to minimize the risk level.
 28. Thenon-transitory computer-readable medium of claim 24 wherein the targetsystem includes one or more financial markets, wherein the plurality ofinter-related elements include items that can be purchased and/or soldin the one or more financial markets, wherein the control elementsinclude one or more controls to determine whether to purchase or sellparticular items at particular times and to select one or more actionsto initiate transactions to purchase or sell the particular items at theparticular times, and wherein the goal includes to maximize profit whilemaintaining risk below a specified threshold.
 29. The non-transitorycomputer-readable medium of claim 24 wherein the target system includesfunctionality to perform coding for medical procedures performed onhumans, wherein the plurality of inter-related elements include aplurality of medical codes corresponding to a plurality of medicalprocedures, wherein the control elements include one or more controls toselect particular medical codes to associate with particular medicalprocedures in specified circumstances, and wherein the goal includes tominimize errors in selected medical codes that cause revenue leakage.30. A system comprising: one or more processors of one or more computingsystems; and one or more modules that, when executed by at least one ofthe one or more processors, cause the one or more processors todetermine one or more control actions to perform as part of controllinga physical system, the determining of the one or more control actionsincluding: obtaining coupled differential equations that represent astate of a physical system for a specified time and that are generatedfrom system information and objective information and sensorinformation, wherein the physical system has a plurality ofinter-related elements and has one or more outputs whose values varybased at least in part on values of one or more manipulatable controlelements of the plurality, wherein the system information is supplied byone or more users to describe the physical system and includesrestrictions involving the plurality of elements, wherein the objectiveinformation identifies a goal to be achieved during controlling of thephysical system, and wherein the sensor information identifies stateinformation for the specified time for at least one element of theplurality; performing a first piecewise linear analysis of the coupleddifferential equations to train a model that describes a state of thephysical system for the specified time and that includes values of atleast some of the plurality of elements for the specified time, whereinthe performing of the first piecewise linear analysis includessimulating effects of manipulating the one or more manipulatable controlelements for each of one or more prior time periods before the specifiedtime while satisfying the goal for the one or more prior time periods;performing a second piecewise linear analysis of the coupleddifferential equations to identify one or more control actions to takein the physical system for the specified time that manipulate values ofthe one or more manipulatable control elements and that provide asolution for the goal for the specified time, wherein the performing ofthe piecewise linear analysis includes updating the model to reflect thesolution for the goal for the specified time period based at least inpart on the identified state information for the specified time; andproviding information about the identified one or more control actions,to enable actions to be taken in the physical system for the specifiedtime to affect the outputs based on the identified one or more controlactions.
 31. The system of claim 30 wherein the one or more modules arepart of a collaborative distributed decision system and include softwareinstructions for execution by the at least one processor, wherein theprovided solution reflected in the updated model is within a thresholdamount of an optimal solution for the goal for the specified time, andwherein the system further comprises one or more effectuators to performthe actions in the physical system for the specified time bymanipulating the values of the one or more manipulatable controlelements in specified manners for the identified one or more controlactions to affect the outputs.
 32. The system of claim 31 wherein theperforming of the first piecewise linear analysis of the coupleddifferential equations to train the model further includes: for a timewindow from an earlier time to the specified time that includes the oneor more prior time periods, dividing the time window into a successionof a plurality of time slices that each, other than a first time sliceof the succession, overlaps at least in part with a prior time slice ofthe succession; evaluating, based on initial state information for theearlier time, an initial version of the coupled differential equationsto identify an initial solution for the goal for the first time slicethat includes simulating effects of manipulating the one or moremanipulatable control elements to one or more initial values, andstoring an initial version of the model that describes the state of thephysical system for the first time slice and includes the simulatedeffects of the manipulating to the one or more initial values; and foreach time slice of the succession after the first time slice, updating aversion of the stored model from the prior time slice to reflect anadditional solution for the goal for the time slice that includessimulating effects of further manipulating the one or more manipulatablecontrol elements to one or more additional values, and wherein thetrained model is a version of the model after the updating to reflectthe additional solution for the goal for a last time slice of thesuccession.
 33. The system of claim 30 wherein the one or more modulesconsist of one or more means for performing the determining of the oneor more control actions to perform as part of controlling the physicalsystem.